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Executive Summary 


What’s New for the 2016-2017 PARCC Technical Report 


The purpose of this technical report is to describe the third operational administration of the 
Partnership for Assessment of Readiness for College and Careers (PARCC) assessments in the 2016-2017 
academic year. The structure and format of this technical report is consistent with the prior technical 
reports with the following changes or updates: 


e Section 5 — Test Taker Characteristics: Participation included students from Colorado, Bureau 
of Indian Education, District of Columbia, Illinois, Maryland, New Jersey, New Mexico, and 
Rhode Island. 

e Section 8 — Reliability: Reliability is provided at the scale score level in addition to the raw score 
level. When reliability is summarized across forms, the minimum and maximum reliability is 
included in the tables. 

e Section 9 — Validity: The mode and device comparability study from the spring 2015 
administration has been updated based on data from the spring 2017 administration. A 
summary of the first phase of the Study of External Validity of PARCC Performance Standards is 
included. 

e Section 10 -IRT Calibration and Scaling: The online and paper mathematics assessments were 
pre-equated for spring 2017. The paper ELA/L assessments were pre-equated for spring 2017. 
The online ELA/L assessments were post-equated for spring 2017. Pre-equated assessments do 
not have the linking results included as these only apply to post-equated assessments. 

e Section 12 — Scale Scores: Details on the pre-equated conversion tables are provided. In 
Appendix 12, cumulative frequency tables for scale score distributions are provided and the 
conversion tables for the performance level setting forms are removed. The prior technical 
report is referenced for obtaining these. 

e Section 13 — Student Growth Measures: This section is new and provides details on the 
calculation of the Student Growth Percentiles reported to the PARCC participants. 


Overview 


PARCC is a state-led consortium creating next-generation assessments that, compared to traditional K— 
12 assessments, more accurately measures student progress toward college and career readiness. The 
PARCC assessments are aligned to the Common Core State Standards (CCSS) and were administered 
operationally for the first time in the 2014—2015 academic year. PARCC comprises assessments in both 
English language arts/literacy (ELA/L) and mathematics in grades 3 through 8 and high school. 


The information provided in this technical report is intended for use by those who evaluate tests, 
interpret scores, or use test results in making educational decisions. It is assumed that the reader has 
technical knowledge of test construction and measurement procedures, as stated in Standards for 
Educational and Psychological Testing (American Educational Research Association [AERA], American 
Psychological Association [APA], and National Council on Measurement in Education [NCME], 2014). The 
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purpose of this technical report is to describe the third operational administration of the PARCC 
assessments in the 2016-2017 academic year and includes the following topics: 


e background and purpose of the assessments; 

e test development of items and forms; 

e test administration, security, and scoring; 

e test taker characteristics; 

e classical item analyses and differential item functioning; 
e reliability and validity of scores; 

e item response theory (IRT) calibration and scaling; 

e performance level setting; 

e development of the score reporting scales and student performance; 
e student growth measures; and 

e quality control procedures. 


Background and Purpose 

Assessments for the first operational administration were constructed in 2014. Eleven states and the 
District of Columbia participated in the first administration of the PARCC assessments during the 2014— 
2015 school year. A small subset of students was tested in fall 2014. ELA/L grades 9, 10, and 11, and 
Algebra |, Geometry, and Algebra II were administered in the fall; these assessments were administered 
on paper only. The majority of students tested during the spring 2015 window when all grades and 
content areas were administered online and on paper. Seven states, the Bureau of Indian Education, and 
District of Columbia participated in the second administration in school year 2015-2016. Not all 
participating states had students testing in all grades. In fall 2015 ELA/L grades 9, 10, and 11, and 
Algebra |, Geometry, and Algebra II were administered online and on paper. The majority of students 
tested during the spring 2016 window when all grades and content areas were administered online and 
on paper. Six states, the Bureau of Indian Education, and District of Columbia participated in the third 
administration in school year 2016-2017. Not all participating states had students testing in all grades. In 
fall 2016 ELA/L grades 9, 10, and 11, and Algebra |, Geometry, and Algebra II were administered online 
and on paper. The majority of students tested during the spring 2017 window when all grades and 
content areas were administered online and on paper. 


The PARCC assessments are designed to achieve several purposes. First, the tests are intended to 
provide evidence to determine whether students are on track for college- and career-readiness. Second, 
the tests are structured to access the full range of CCSS and measure the total breadth of student 
performance. Finally, the tests are designed to provide data to help inform classroom instruction, 
student interventions, and professional development. 
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Item Types 

The tests contain selected response, brief and extended constructed response, technology-enabled, and 
technology-enhanced items (TEI), as well as performance tasks. Technology-enabled items are single- 
response or constructed-response items that involve some type of digital stimulus or open-ended 
response box with which the students engage in answering questions. Technology-enhanced items 
involve specialized student interactions for collecting performance data. Therefore, the act of 
performing the task is the way in which data are collected. Students may be asked, among other tasks, 
to categorize information, organize or classify data, order a series of events, plot data, generate 
equations, highlight text, or fill in a blank. One example of a TEI is an interaction in which students are 
asked to drag response options onto a Venn diagram to show the relationship among ideas. 


Pre- and Post-Equating 

Spring 2017 PARCC assessments (except ELA/L online) were pre-equated, meaning that the scoring 
tables were based on item parameters estimated using data from earlier administrations. The ELA/L 
online test continued to be post-equated due to the structure of these tests. ELA/L paper tests used only 
established operational items to make pre-equating possible. Mathematics tests have embedded field- 
test items that provide reliable statistics for new items. This coupled with the evolution of the item bank 
given the longevity of the program made pre-equating possible. Pre-equating allows for faster reporting 
of results. In this technical report, item statistics are from the administration the item parameters were 
estimated that were used to score the students. 


Classical and IRT Item Analysis 

Classical item analyses and differential item functioning analyses were performed on the data to 
evaluate the psychometric characteristics of the operational test items after items were administered 
and before scores were reported. The two-parameter logistic/generalized partial credit (2PL/GPC) IRT 
model was used for calibrations and scaling. Multiple operational core forms were administered for 
each grade in ELA/L and mathematics. The forms included sets of embedded common items to provide 
data to support horizontal linking across test forms within a grade and content area and for post- 
equated tests across years. Forms that were pre-equated, meaning that the conversion table is based on 
item parameters estimated using data from earlier operational administrations, have parameter 
estimates already on the base IRT scale, so there was no need to create year-to-year common item 
(linking) sets for these grades/subjects. 


For post-equated ELA/L online tests, IRT calibration and scaling placed all operational items for a single 
grade onto a common scale. After calibration and model fit evaluation was completed, a master list of 
all items flagged as problematic was compiled and brought to the PARCC Priority Alert Task Force.’ The 
task force reviewed each item, its content and the statistical properties, and made decisions about 
whether to include the item in the operational scores. Sometimes, an item was rejected because it 
appeared to have content issues, and sometimes an item was excluded because it could not be 
calibrated or showed extremely poor IRT model fit. Ultimately the decision about whether to keep or 
exclude each flagged item was made by the task force. The goals of the task force were to: a) minimize 


1 The Priority Alert Task Force comprised Parcc Inc. staff, state leads and content experts, and Pearson staff. 
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the number of items excluded from the operational test forms, and b) avoid advantaging or 
disadvantaging any test takers. 


In response to several practical constraints based on the number of forms constructed for each mode 
and to meet the blueprints (e.g., inclusion of TEI on computer-based testing [CBT] forms), there was no 
single CBT form that was administered intact in the paper delivery mode at any grade level. For 
example, TEI from online forms were replaced in the paper forms with items having similar content, but 
appropriate for paper-based testing (PBT). However, for both ELA/L and mathematics, the content on 
PBT forms significantly overlapped content on the CBT forms. Mode comparability studies were 
conducted in 2015 and 2017; the results are presented in separate technical reports. The studies 
evaluated the extent to which scores from CBT and PBT forms could be considered as comparable with 
regard to psychometric characteristics. A major finding was that score comparability was inconsistent 
across the content domains and grade levels investigated. 


Overall Scale Scores, Claim Scores, and Subclaim Scores 

The PARCC ELA/L and mathematics scores are expressed as various types of scale scores (both total 
scores and claim scores, related to the claims structures described below), as well as by performance 
levels used to describe how well students meet the academic standards for their grade level. On the 
basis of a student’s total score, an inference is drawn about how much knowledge and skill in the 
content area the student has acquired. The total score is also used to classify students in terms of the 
level of knowledge and skill in the content area as students progress in their K-12 education. These 
levels are called performance levels and are reported as: 


e Level 5: Exceeded expectations 

e Level 4: Met expectations 

e Level 3: Approached expectations 

e Level 2: Partially met expectations 

e Level 1: Did not yet meet expectations 


Students classified as either Level 4 or Level 5 are meeting or exceeding the grade level expectations. 
Additionally, information on more specific skills is provided and is reported as Below Expectations, 
Nearly Meets Expectations, and Meets or Exceeds Expectations. 


PARCC developed performance level descriptors (PLDs) to assist with the understanding and 
interpretations of the ELA/L and mathematics scores (https://parcc-assessment.org/performance- 
levels/). Additionally, resource information is available online to educators, parents, and students 
(http://understandthescore.org/score-report-guide/), which includes information on understanding and 
interpreting the ELA/L and mathematics score reports. 


The claim structures for ELA/L and mathematics, grounded in the Common Core State Standards, inform 
the design and development of the summative assessments. 
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Claim Structure for ELA/L 
Master Claim. The master claim is the overall performance goal for the PARCC ELA/L 
Assessment System—students must demonstrate that they are college- and career-ready or on 
track to readiness as demonstrated through reading and comprehending of grade-level texts of 
appropriate complexity and writing effectively when using and/or analyzing sources. 


Major Claims: 1) reading and comprehending a range of sufficiently complex texts 
independently, and 2) writing effectively when using and/or analyzing sources. 


Subclaims: The subclaims further explicate what is measured on the PARCC assessments and 
include claims about student performance on the standards and evidences outlined in the 
PARCC evidence tables for reading and writing (https://parcc-assessment.org/ela-literacy). The 
claims and evidences are grouped into the following categories: 


Vocabulary, Interpretation, and Use 
Reading Literature 

Reading Informational Text 

Written Expression 


+ i aa ee 


Knowledge of Language and Conventions 


Claim Structure for Mathematics 
Master Claim. The degree to which a student is college- or career-ready or on track to being 
ready in mathematics. The student solves grade-level/course-level problems aligned to the 
Standards for Mathematical Content with connections to the Standards for Mathematical 
Practice. 


Subclaims: The subclaims further explicate what is measured on the PARCC assessments and 
include claims about student performance on the standards and evidences outlined in the 
PARCC evidence statement tables for mathematics (https://parcc- 
assessment.org/mathematics/). The claims and evidence are grouped into the following 
categories: 


Subclaim A: Major Content with Connections to Practices 
Subclaim B: Additional and Supporting Content with Connections to Practices 


Subclaim C: Highlighted Practices with Connections to Content: Expressing mathematical 
reasoning by constructing viable arguments, critiquing the reasoning of others, and/or 
attending to precision when making mathematical statements 


Subclaim D: Highlighted Practice with Connections to Content: Modeling/Application by 
solving real-world problems by applying knowledge and skills articulated in the standards 
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Scale Scores 

Scale scores were defined for each test as a linear transformation of the IRT theta (8) scale. The test 
characteristic curves associated with the performance level setting forms were used to identify the theta 
values associated with the Level 2 and Level 4 point scores. By defining Level 2 and Level 4 scale scores 
to be 700 and 750, respectively, the linear relationship between theta and scale scores was established. 


The result is 201 defined full summative scale score points for each ELA/L and mathematics assessment, 
ranging from 650 to 850. A scale score of 700 is always the minimum for Level 2 performance, and a 
scale score of 750 is always the minimum for Level 4 performance. 


The thresholds for summative performance levels on the scale score metric recommended by the scale 
score task force are described in Table 1.1. 


Table 1.1 Defined Summative Scale Scores and Cut Scores 


Lowest Highest 
: Cut Score Cut Score : 
Obtainable Scale Obtainable Scale 
Level 2 Level 4 
Score Score 
Full Summative 650 700 750 850 


As with the full summative scores, scale scores for reading and writing were defined for each test as a 
linear transformation of the IRT theta (8) scale. The same IRT theta scale was used for reading and 
writing as was used for the ELA/L full summative scores. The theta values associated with the Level 2 
and Level 4 performance levels were identified using the test characteristic curves associated with the 
performance level setting forms. Parallel to the full summative scores, the relationship between theta 
and scale scores was established with Level 2 and Level 4 theta scores and the corresponding predefined 
scale scores. 


The result was 81 defined scale score points for reading, ranging from 10 to 90. A scale score of 30 is the 
cut score for minimum Level 2 performance, and a scale score of 50 is the cut score for minimum Level 4 
performance. There are 51 defined scale score points for writing, ranging from 10 to 60. A scale score of 
25 is the cut score for minimum Level 2 performance, and a scale score of 35 is the cut score for 
minimum Level 4 performance. The threshold reading and writing performance levels on the scale score 
metric recommended by the scale score task force are described in Table 1.2. 


Table 1.2 Defined Scaled Scores and Cut Scores for Reading and Writing Claim Scores 


Lowest Highest 
. Cut Score Cut Score ‘ 
Obtainable Scale Obtainable Scale 
Level 2 Level 4 
Score Score 
Reading 10 30 50 90 
Writing 10 25 35 60 
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Regarding the subclaim scores, the Level 4 cut is defined as Meets or Exceeds Expectations because 
grade 3-8 and high school students at Level 4 or above are likely to have the skills and knowledge to 
meet the definition of career and college readiness. Subclaim outcomes center on that performance 
level and are reported as Below Expectations, Nearly Meets Expectations, and Meets or Exceeds 
Expectations. 


Quality Control 

To ensure IRT calibrations, scaling and conversion tables were produced accurately, HumRRO replicated 
the data processing, IRT calibrations and scale score transformations carried out by Pearson, and the 
generation of the score conversion tables. Pearson and HumRRO independently generated incomplete 
data matrices and conducted the calibrations using IRTPRO (Cai, Thissen & du Toit, 2011) calibration 
software. Pearson’s scaling constants were compared to those generated by HumRRO and found to be 
consistent. Measured Progress (MP) performed independent quality control comparisons between the 
Pearson and HumRRO item parameter estimates to identify any differences. In addition, MP 
independently made certain that the same items were excluded from the linking sets, and compared 
transformed parameter estimates computed by Pearson and HumRRO. If items had large differences 
across years or modes, the items were discussed and any remaining issues resolved. MP prepared 
reports documenting their findings. Exact matches were found between all Pearson and HumRRO 
conversion tables before scores were reported. 
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Section 1: Introduction 


1.1 Background 


States associated with the Partnership for Assessment of Readiness for College and Careers (PARCC) 
came together in early 2010 with a shared vision of ensuring that all students—regardless of income, 
family background, or geography—have equal access to a world-class education that will prepare them 
for success after high school in college and/or careers. The PARCC goal was to develop new assessments 
that tie into more rigorous academic expectations and help prepare students for success in college and 
the workforce, as well as to provide information back to teachers and parents about where students are 
on their path to success. Calling on the expertise of thousands of teachers, higher education faculty, and 
other educators in multiple states, the PARCC assessment system is a high-quality set of summative 
assessments, diagnostic assessments, formative tasks, and other support materials for teachers 
including professional development and communications tools. 


The PARCC consortium develops and administers next-generation assessments that, compared to 
traditional K-12 assessments, more accurately measure student progress toward college and career 
readiness. The assessments are aligned to the Common Core State Standards (CCSS) and include both 
English language arts/literacy (ELA/L) assessments (grades 3 through 11) and mathematics assessments 
(grades 3 through 8 and high school). Compared to traditional standardized tests, these assessments are 
intended to measure more complex skills like critical-thinking, persuasive writing, and problem-solving. 


In 2013, the PARCC Governing Board launched Parcc Inc., a non-profit organization designed to support 
the successful delivery of the tests in 2014—15, and the long-term success of the multi-state 

partnership. States continue to govern decisions about the assessment system; the non-profit 
organization is their “agent” for overseeing the many vendors involved in the PARCC assessment system, 
coordinating the multiple work groups and committees (including Governing Board meetings), managing 
the PARCC intellectual property, overseeing the research agenda and the Technical Advisory Committee, 
and developing and launching the multiple non-summative tools. 


Summative assessments for the first operational administration were constructed in 2014. Eleven states 
and the District of Columbia participated in the first administration of the PARCC assessments during the 
2014-2015 school year. A small subset of students tested in the fall 2014 window. ELA/L grades 9, 10, 
and 11, and Algebra |, Geometry, and Algebra II were administered in the fall; these assessments were 
administered on paper only. The majority of students tested during the spring 2015 window when all 
grades and content areas were administered online and on paper. Seven states, the Bureau of Indian 
Education, and District of Columbia participated in the second administration in school year 2015-2016. 
Not all participating states had students testing in all grades. In the fall 2015 window, ELA/L grades 9, 10, 
and 11, and Algebra |, Geometry, and Algebra II were administered online and paper. The majority of 
students tested during the spring 2016 window when all grades and content areas were administered 
online and on paper. 
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The purpose of this technical report is to describe the third operational administration of the PARCC 
summative assessments in the 2016-2017 academic year, including test form construction, test 
administration, item scoring, test taker characteristics, classical item analysis results, reliability results, 
evidence of validity, item response theory (IRT) calibrations and scaling, performance level setting 
procedure, growth measures, and quality control procedures. 


1.2 Purpose of the Operational Tests 


The PARCC assessments are designed to achieve several purposes. First, the tests are intended to 
provide evidence to determine whether students are on track for college- and career-readiness. Second, 
the tests are structured to access the full range of CCSS and measure the total breadth of student 
performance. Finally, the tests are designed to provide data to help inform classroom instruction, 
student interventions, and professional development. 


1.3 Composition of Operational Tests 


Each operational test form was constructed to reflect the full test blueprint in terms of content, 
standards measured, and item types. Sets of common items, included to provide data to support 
horizontal linking across test forms within a grade and content area, were proportionally representative 
of the operational test blueprint. 


The current PARCC assessments were administered in either computer-based (CBT) or paper-based 
(PBT) format. English language arts/literacy (ELA/L) assessments focused on writing effectively when 
analyzing text. Mathematics assessments focused on applying skills and concepts, and understanding 
multi-step problems that require abstract reasoning and modeling real-world problems, precision, 
perseverance, and strategic use of tools. In both content areas, students also demonstrated their 
acquired skills and knowledge by answering selected response items and fill-in-the-blank questions. 


PARCC is a mixed-format test. The ELA/L assessment has between two and four literary and 
informational texts; each text has five or six brief comprehension and vocabulary questions. ELA/L 
constructed-response items include three types of tasks: literary analysis, narrative writing, and research 
simulation. For each task, students are instructed to read one or more texts, answer several brief 
questions, and then write an essay based on the material they read. The mathematics assessment 
contains tasks that measure a combination of conceptual understanding, applications, skills, and 
procedures. Mathematics constructed-response items consist of tasks designed to assess a student’s 
ability to use mathematics to solve real-life problems. Some of the tasks require students to describe 
how they solved a problem, while other tasks measure conceptual understanding and ability to apply 
concepts by means of selected-response or technology-enhanced items. In addition, students are 
required to demonstrate their skills and knowledge by answering innovative selected-response and 
short-answer questions that measure concepts and skills. 


Each assessment consisted of multiple units, and additionally, one of the mathematics units was split 
into two sections: a non-calculator section and a calculator section. 
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1.4Intended Population 


The PARCC tests are intended for students taking ELA/L and/or mathematics in grades 3 through 11, as 
well as students taking high school mathematics (i.e., Algebra |, Geometry, Algebra Il, Integrated 
Mathematics I-III). For these students, the PARCC tests measured whether students were meeting state 
academic standards and mastering the knowledge and skills needed to progress in their K-12 education 
and beyond. 


1.5Groups and Organizations Involved with PARCC 


Parcc Inc. is a nonprofit organization that assumes the responsibility for management of the PARCC 
consortium, as well as the development and implementation of PARCC assessments. 


A number of committees of educators, state education agency staff, and national experts lead the work 
of the PARCC consortium. These committees include: 


e the PARCC consortium Governing Board that makes major policy and operational decisions; 

e the Technical Advisory Committee that helps ensure all assessments will provide reliable results 
to inform valid instructional and accountability decisions; 

e the K-12 State Leads that coordinate all aspects of development of the PARCC assessment 
system and serve as the conduit to the Technical Advisory Committee and the Governing 
Board; 

e the Advisory Committee on College Readiness, which includes higher education executive 
officers from PARCC states and other state- and nationally recognized leaders in the 
postsecondary community; and 

e the Higher Education Leadership Team, which is responsible for coordinating higher education 
engagement in the PARCC assessment system and works closely with the Advisory Committee 
on College Readiness. 


Test and item development activities were conducted by Pearson and WestEd under the guidance and 
oversight of PARCC leadership.” 


Pearson served as the primary contractor for the PARCC operational administration and was responsible 
for developing test forms, producing all testing materials, packaging and distribution, receiving and 
scanning of materials, and scoring, as well as program management and customer service. 


Pearson Psychometrics was responsible for all psychometric analyses of the PARCC operational test 
data. This included classical item analyses, differential item functioning (DIF) analyses, item calibrations 
based on item response theory (IRT), scaling, and development of all conversion tables. 


 PARCC leadership includes the following groups: PARCC Governing Board, K-12 State Leads, Higher Education 
Leadership Team, Technical Advisory Committee, Operational Working Group members from each of the member 
states, and staff members from Parcc, Inc., the project management partner for the PARCC Consortium. 
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HumRRO served as a subcontractor and was responsible for replicating item calibrations based on item 
response theory (IRT), scaling, and development of all conversion tables. 


Measured Progress (MP) served as a subcontractor to conduct external evaluations; they were 
responsible for reviewing and comparing the psychometric IRT calibrations performed by Pearson, 
which were replicated by HumRRO. MP also provided comparisons of results obtained independently 
from Pearson and from HumRRO for conversion tables, summative and claim scale scores, performance 
level classifications, and subclaim performance level classifications. 


1.6 Overview of the Technical Report 


This report begins by providing explanations of the test form construction process, test administration, 
and scoring of the test items. Subsequent sections of the report present descriptions of test taker 
characteristics, results of classical item analyses, results of reliability analyses, evidence of validity, item 
response theory (IRT) calibrations and scaling, performance level setting procedure, measures of 
student growth, and quality control procedures. 


The technical report contains the following sections: 
e Section 2 —Test Development 


This section describes the PARCC test design and the procedures followed during the 
development of operational test forms. 


e Section 3 — Test Administration 


This section presents the operational administration schedule, information regarding test 
security and confidentiality, accessibility features and accommodations, and testing 
irregularities and security breaches. 


e Section 4 — Scoring of the Items 


The key-based and rule-based processes for machine scored items, as well as the training and 
monitoring processes for human scored items are provided in this section. 


e Section 5 — Test Taker Characteristics 


This section describes the composition of test forms, rules for inclusion of students in analyses, 
distributions of test takers by grade, mode, and gender, and distributions of demographic 
variables of interest. 
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Section 6 — Classical Item Analyses 


The classical item-level statistics calculated for the operational test data, the flagging criteria 
used to identify items that performed differently than expected, and the results of these 
analyses are presented in this section. 


Section 7 — Differential Item Functioning 


In this section, the methods for conducting differential item functioning analyses as well as 
corresponding flagging criteria are described. This is followed by definitions of the comparison 
groups and subsequent results for the comparison groups. 


Section 8 — Reliability 


The results of internal consistency reliability analyses and corresponding standard errors of 
measurement, for each grade, content area, and mode (CBT or PBT) for all test takers, and for 
subgroups of interest, is provided in this section. This is followed by reliability results for 
subscores and reliability of classification (i.e., decision accuracy and decision consistency). 
Finally, expectations and results for interrater agreement for hand scored items are 
summarized. 


Section 9 — Validity 


Validity evidence based on analyses of the internal structure of the tests is provided in this 
section. Correlations between subscores are reported by grade, content area, and mode (CBT or 
PBT) for all test takers. 


Section 10 -IRT Calibration and Scaling 


This section presents the information related to the calibration and scaling of item response 
data including: data preparation, the calibration process, model fit evaluation, and items 
excluded from score reporting. In addition, the scaling process is described and evaluated. 


Section 11 — Performance Level Setting (PLS) Procedure and Results 


Performance levels and policy definitions, as well as the processes followed to establish 
performance level thresholds are described in this section. 


Section 12 — Scale Scores 


This section provides an overview of the claims and subclaims, describes the development of the 
reporting scales and conversion tables, and presents scale score distributions. Finally, 
information regarding the interpretation of claim scores and subclaim scores is presented. 
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e Section 13 — Student Growth Measures 


This section provides details on student growth percentiles (SGP). Information about the model, 
model fit, and SGP averages at the consortium-level for all test takers, and for subgroups of 
interest, are provided in this section. 


e Section 14 — Quality Control Procedures 


All aspects of quality control are presented in this section. These activities range from quality 
assurance of item banking, test form construction, and all testing materials to quality control of 
scanning, image editing, and scoring. This is followed by a detailed description of the steps taken 
to ensure that all psychometric analyses were of the highest quality. 


e References 
e Appendices 


To facilitate utility, tables in the appendices are numbered sequentially according to the section 
represented by the tables. For example, the first appendix table for Section 5 is numbered A.5.1, 
the second appendix table for Section 5 is numbered A.5.2, and so on. 


e Addendum 


The addendum presents the results of analyses for the fall operational administration. These 
results are reported separately from the spring results because fall testing involved a 
nonrepresentative subset of students testing only ELA/L grades 9, 10, and 11, as well as Algebra 
|, Geometry, and Algebra Il. 


To organize the addendum, tables are numbered sequentially according to the section 
represented by the tables. For example, the first addendum table for Section 5 is numbered 
ADD.5.1, the second addendum table for section 5 is numbered ADD.5.2, and so on. 
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1.7 Glossary of Abbreviations 


Table 1.3 Glossary of PARCC Abbreviations and Acronyms 


Abbreviation/Acronym 


Definition 


1PL/PC One-parameter/Partial Credit Model 

2PL/GPC Two-parameter Logistic/Generalized Partial Credit Model 
3PL/GPC Three-parameter Logistic/Generalized Partial Credit Model 
AAF Accessibility, Accommodations, and Fairness 

ABBI Assessment Banking for Building and Interoperability 
AERA American Educational Research Association 

AlS Average Item Score 

AlQ Assessment and Information Quality 

APA American Psychological Association 

ASC Additional and Supporting Content (Mathematics) 
ASL American Sign Language 

ATA Automatic Test Assembler 

CBT Computer-Based Test 

CCSS Common Core State Standards 

CDQ Customer Data Quality 

CSEM Conditional Standard Error of Measurement 

DIF Differential Item Functioning 

DPL Digital Production Line 

DPP Digital Pre-press 

EBSS Evidence-based Standard Setting 

ELA/L English Language Arts/Literacy 

EL English Learners 

EOC End-of-Course 

EOY End-of-Year 

ePEN2 Electronic Performance Evaluation Network second generation 
ESEA Elementary and Secondary Education Act 

FRL Free or Reduced-price Lunch 

FS Full Summative 

FT Field Test 

IA Item Analysis 

ICC Item Characteristic Curve 

IDEA Individuals with Disabilities Education Act 

IEP Individualized Education Program 

INF Information Curve 

IRA Inter-rater Agreement 

IRF Item Response File 

IRT Item Response Theory 

ISR Individual Student Report 

K-12 Kindergarten to Grade 12 

LEA Local Education Agency 

LID Local Item Dependence 

MAD Mean Absolute Difference 

MC Major Content (Mathematics) 

MH Mantel-Haenszel 

MP Measured Progress 
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Abbreviation/Acronym Definition 

MP Modeling Practice (Mathematics) 

MR Mathematical Reasoning 

NAEP National Assessment of Educational Progress 
NCLB No Child Left Behind 

NCME National Council on Measurement in Education 
NSLP National School Lunch Program 


OE responses 


Open-ended responses 


OMR 


Optical Mark Reading 


OWG Operational Working Group 

PARCC Partnership for Assessment of Readiness for College and Careers 
PBA Performance-Based Assessment 
PBT Paper-Based Test 

PCR Prose Constructed Response (ELA/L) 
PEJ Postsecondary Educators’ Judgment 
PLD Performance Level Descriptor 

PLS Performance Level Setting 

PV Product Validation 

QA Quality Assurance 

RD Reading (ELA/L) 

RI Reading Information (ELA/L) 

RL Reading Literature (ELA/L) 

RMSD Root Mean Square Difference 

RV Reading Vocabulary (ELA/L) 

RST Raw-score-to-theta 

SD Standard Deviation 

SDF Student Data File 

SE Standard Error 

SEJ Standard Error of Judgment 

SEM Standard Error of Measurement 
SIRB Scored Item Response Block 

SMD Standardized Mean Difference 
SSMC Single Select Multiple Choice 

SWD Students with Disabilities 

TCC Test Characteristic Curve 

TTS Text to Speech 

UIN Unique Item Number 

WE Writing Written Expression (ELA/L) 
WKL Writing Knowledge Language and Conventions (ELA/L) 
WLS Weighted Least Squares 

WR Writing (ELA/L) 

WRMSD Weighted Root Mean Square Difference 
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Section 2: Test Development 


2.1 Overview of the PARCC Assessment, Claims, and Design 


Aligned to the Common Core State Standards (CCSS) as articulated in the PARCC Model Content 
Frameworks, the PARCC assessments are designed to determine whether students are college- and 
career-ready or on track, assess the full range of the CCSS, measure the full range of student 
performance, and provide data to help inform instruction, interventions, and professional development. 
Test development is an ongoing process involving educators, researchers, psychometricians, subject 
matter professionals, and assessment experts who participate in the development of the PARCC test 
design and its underlying foundational documents; develop and review passages and items used to build 
the PARCC assessments; monitor the program for quality, accessibility, and fairness for all students; and 
construct, review, and score the assessments 


The PARCC summative assessments include both English language arts/literacy (ELA/L) and mathematics 
assessments in grades 3 through 8 and high school. The high school mathematics tests include 
traditional mathematics and integrated mathematics course pathways. Tests contain selected response, 
brief and extended constructed response, technology-enabled and technology-enhanced items (TEI), as 
well as performance tasks. Technology-enabled items are single-response or constructed-response 
items that involve some type of digital stimulus or open-ended response box with which the students 
engage in answering questions. Technology-enhanced items involve specialized student interactions for 
collecting performance data. In other words, the act of performing the task is the way in which data is 
collected. Students may be asked, among other interactions, to categorize information, organize or 
classify data, order a series of events, plot data, generate equations, highlight text, or fillin a blank. One 
example of a TEI is an interaction in which students are asked to drag response options onto a Venn 
diagram to show the relationship among ideas. 


The PARCC assessments offer a wide range of accessibility features for all students and accommodations 
for students with disabilities (e.g., screen reader, assistive technology, braille, large print [LP], text-to- 
speech [TTS], and American Sign Language [ASL] video versions of the test, as well as response 
accommodations that allow students to respond to test items using different formats). For English 
learners who are native Spanish speakers, PARCC offers the mathematics assessments in Spanish, and 
both LP and TTS versions of the test in Spanish (refer to the PARCC Accessibility Features and 
Accommodations Manual for in-depth information). 


2.1.1 English Language Arts/Literacy (ELA/L) Assessments—Claims and Subclaims 


The ELA/L summative assessment at each grade level consists of three task types: literary analysis, 
research simulation, and narrative writing. For each performance-based task, students are asked to read 
or view one or more texts, answer comprehension and vocabulary questions, and write an extended 
response that requires them to draw evidence from the text(s). The summative assessment also 
contains literary and informational reading passages with comprehension and vocabulary questions. 
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The claim structure, grounded in the CCSS, undergirds the design and development of the ELA/L 
summative assessments. 


Master Claim. The master claim is the overall performance goal for the PARCC ELA/Literacy 
Assessment System—students must demonstrate that they are college- and career-ready or on 
track to readiness as demonstrated through reading and comprehending of grade-level texts of 
appropriate complexity and writing effectively when using and/or analyzing sources. 


Major Claims: 1) reading and comprehending a range of sufficiently complex texts 
independently, and 2) writing effectively when using and/or analyzing sources. 


Subclaims: The subclaims further explicate what is measured on the PARCC assessments and 
include claims about student performance on the standards and evidences outlined in the 
PARCC evidence tables for reading and writing (refer to PARCC test specifications documents). 
The claims and evidences are grouped into the following categories: 


Vocabulary Interpretation and Use 
Reading Literature 

Reading Informational Text 
Written Expression 


Oia: er 


Knowledge of Language and Conventions 
2.1.2 Mathematics Assessments—Claims and Subclaims 


The summative mathematics assessment at each grade level includes both short- and extended- 
response questions focused on applying skills and concepts to solve problems that require 
demonstration of the mathematical practices from the CCSS with a focus on modeling and reasoning 
with precision. The assessments also include performance-based short-answer questions focused on 
conceptual understanding, procedural skills, and application. 


The claim structure, grounded in the CCSS, undergirds the design and development of the summative 
assessments. 


Master Claim. The degree to which a student is college- or career-ready or on track to being 
ready in mathematics. The student solves grade-level/course-level problems aligned to the 
Standards for Mathematical Content with connections to the Standards for Mathematical 
Practice. 


Subclaims: The subclaims further explicate what is measured on the PARCC assessments and 
include claims about student performance on the standards and evidences outlined in the 
PARCC evidence statement tables for mathematics (refer to PARCC test specifications 
documents). The claims and evidence are grouped into the following categories. 


Subclaim A: Major Content with Connections to Practices 
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Subclaim B: Additional and Supporting Content with Connections to Practices 


Subclaim C: Highlighted Practices with Connections to Content: Expressing mathematical 
reasoning by constructing viable arguments, critiquing the reasoning of others, and/or attending 
to precision when making mathematical statements 


Subclaim D: Highlighted Practice with Connections to Content: Modeling/Application by solving 
real-world problems by applying knowledge and skills articulated in the standards 


2.2 Test Development Activities 


Test development activities began with the standards and model content frameworks. From these, 
PARCC, in collaboration with more than 2,000 educators, researchers, and psychometricians, has 
developed the PARCC test specifications documents that guide the development of test items and the 
composition of the tests. These documents include the College- and Career-Ready Determinations and 
Performance-Level Descriptions, Claim Structure, Evidence Statement Tables, Blueprints, Informational 
Guides, Passage Selection Guidelines, Mathematics Sequencing Guidelines, Task Generation Models, 
Fairness and Sensitivity Guidelines, Text Selection Guidelines, and the Style Guide. Refer to the PARCC 
website for further information about these documents. 


2.2.1 Item Development Process 


PARCC test and item development activities were conducted by Pearson and WestEd under the 
guidance and oversight of PARCC leadership, including the PARCC Governing Board, the K-12 state 
leads, the Higher Education Leadership Team, the Technical Advisory Committee, the Operational 
Working Group members from each of the member states, the PARCC State Text and State Content Item 
Review Committees, and staff members from Parcc, Inc., the project management partner for the 
PARCC Consortium. 


Developing high quality assessment content with authentic stimuli for computer-based tests (CBT) and 
paper-based tests (PBT) measuring rigorous standards is a complex process involving the services of 
many experts including assessment designers, psychometricians, managers, trainers, content providers, 
content experts, editors, artists, programmers, technicians, human scorers, advisors, and members of 
the PARCC Operational Working Groups (OWGs). 


Bank Analysis and Item Development Plan 

The PARCC summative item bank houses passages and items at each assessed grade level and subject. 
The bank supports the administration of the assessments, along with item release and practice tests. 
Items are developed and field tested annually. Prior to the annual item development cycle, the item 
development teams, in conjunction with members of the OWGs for ELA/L and mathematics, evaluated 
the strengths of the bank and considered the needs for future tests to establish an item development 
plan. 
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Text Selection for ELA/L 

Using the PARCC Passage Selection Guidelines, English language arts subject matter experts were 
trained to search for appropriate passages to support an annual pool of passages for consideration. 
Guided by the PARCC test specifications documents, Pearson and WestEd recruited, trained, and 
managed the contracted subject matter experts to deliver the number of texts specified in the annual 
item development plan. The Passage Selection Guidelines provided a text complexity framework and 
guidance on selecting a variety of text types and passages that allow for a range of standards/evidences 
to be demonstrated to meet the PARCC claims. PARCC ELA/L tests are based on authentic texts, 
including multi-media stimulus. Authentic texts are grade-appropriate texts that are not developed for 
the purposes of the assessment or to achieve a particular readability metric, but reflect the original 
language of the authors. Pearson and WestEd content experts reviewed the passages for adherence to 
the PARCC Passage Selection Guidelines to meet the annual item development plan described above in 
the number and distribution of genres and topics prior to review and consideration by the State Text 
Review Committee. ELA/L item development was not conducted until after texts were approved by the 
State Text Review Committee. 


Item Development 

Guided by the PARCC foundational documents, Pearson and WestEd recruited and trained the item 
writers and managed the item writing to develop the number of items specified in the annual item 
development plan. Prior to further committee reviews, the assessment teams at Pearson and WestEd 
reviewed the items for content accuracy, alignment to the standards, range of difficulty, adherence to 
universal design principles (which maximize the participation of the widest possible range of students), 
bias and sensitivity, and copy edit to enable the accurate measurement of the PARCC standards. 


2.2.2 Item and Text Review Committees 


Members of the PARCC OWGs for ELA/L and mathematics, state-level experts, local educators, post- 
secondary faculty, and community members from the PARCC states conducted rigorous reviews of every 
item and passage being developed for the PARCC assessment system to ensure all test items are of the 
highest quality, aligned to the standards, and fair for all student populations. All PARCC reviewers were 
nominated by their state education agency. The purpose of the educator reviews was to provide 
feedback to Pearson, WestEd, and PARCC on the quality, accuracy, alignment, and appropriateness of 
the test passages and items developed annually for the summative PARCC assessments. The meetings 
were conducted either in person or virtually and included large group training on the expectations and 
processes of each meeting, followed by breakout meetings of grade/subject working committees where 
additional training was provided. 
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State Text Review 

The State Text Review is a review and approval by the State Text Review Committee of the texts eligible 
for item development. Participants reviewed and provided feedback to Pearson, WestEd, and PARCC 
about the grade-level appropriateness, content, and potential bias concerns, and reached consensus 
about which texts would move forward for development. The State Text Review Committee was made 
up of members of both State Content Item Review and Bias and Sensitivity Review Committees. 


State Content Item Review 

During State Content Item Review, committees reviewed and edited test items for adherence to the 
PARCC foundational documents, basic universal design principles, PARCC Accessibility Guidelines, 
associated item metadata, and the PARCC Style Guide. Committees accessed the item content within 
the Pearson Assessment Banking for Building and Interoperability (ABBI) system that previews how the 
passages and items will be displayed in an operational online environment. Committees also verified 
that the appropriate scoring rule had been applied to each item. The State Content Item Review 
Committees were made up of OWG members and educators nominated by PARCC member states. 


State Bias and Sensitivity Review 

Educators and community members make up the committee that reviews items and tasks to confirm 
that there are no bias or sensitivity issues that would interfere with a student’s ability to achieve his or 
her best performance. The committee reviewed items and tasks to evaluate adherence to the Fairness 
and Sensitivity Guidelines, and to ensure that items and tasks do not unfairly advantage or disadvantage 
one student or group of students over another. Bias and Sensitivity Committee members made edits 
and modifications to items and passages to eliminate sources of bias and improve accessibility for all 
students. 


Editorial Review 

The PARCC Editorial Review Committee consists of state-level editors who reviewed up to 10 percent of 
the items and tasks. The committee reviewed the items for grammar, punctuation, clarity, and 
adherence to the PARCC Style Guide. 


Data Review 

Following the field test, educator and bias committee members met to evaluate test items and 
associated performance data with regard to appropriateness, level of difficulty, and potential gender, 
ethnic, or other bias, then recommended acceptance or rejection of each field-test item for inclusion on 
an operational assessment. The Data Review Committee also made recommendations that items be 
revised and re-field tested. Items that were approved by the committee are eligible for use on 
operational summative assessments. 


2.2.3 Operational Test Construction 


Under the guidance in the operational test form creation specifications, Pearson constructed the 
operational forms to adhere to the test blueprints and the assessment goals outlined in the form 
creation specifications. These goals were: 
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e test forms designed to measure well across the full range of student ability; 

e scores that are comparable among forms and across test administrations; 

e scales that support classification of students into performance levels; 

@ maximization of the number of parallel forms; 

e minimization of overexposure of items; and 

e adherence to standards for validity, reliability, and fairness (Standards for Educational and 
Psychological Testing, 2014). 


Each content-area and grade-level assessment was based ona specific test blueprint that guided how 
each test was built. Test blueprints determined the range and distribution of content, and the 
distribution of points across the PARCC subclaims and task types. 


Multiple operational forms were constructed for each grade/subject. These forms were designed to 
facilitate psychometric equating through a common item linking strategy (described in Section 2.2.4) 


|’ 


and to be constructed as “parallel” as possible from a content and test-taking experience. Evaluation 
criteria for parallelism included adherence to blueprint; sequencing of content across the forms; 
statistical averages and distributions for difficulty (e.g., p value) and discrimination (e.g., polyserial 
correlation); item type and cognitive complexity; and passage characteristics for ELA/L including genre, 


topics, word count, and text complexity. 


Core forms are the operational test forms consisting of only those items that will count toward a 
student’s score. Core forms are constructed to meet the blueprint and psychometric properties outlined 
in the test construction specifications. PARCC creates multiple core forms for a given assessment to 
enhance test security and to support opportunity for item release. The number of core operational 
forms per grade/subject is provided in Table 2.1. Additionally, appropriate forms were identified as 
accessibility and accommodated forms, and the core forms for all mathematics assessments included 
embedded field-test items. A sample of students were administered ELA/L core forms with embedded 
field-test items. Accessibility and accommodated forms and embedded field testing are described later 
in this section. 


March 27, 2018 Page 37 


PARCC 2017 Technical Report 


Table 2.1 Number of Core Operational Forms per Grade/Subject and Mode for ELA/L and Mathematics* 


: ELA/L Mathematics 
Grade/Subject 
CBT PBT CBT PBT 

Grade 3 3 1 3 1 
Grade 4 3 1 3 1 
Grade 5 3 1 3 1 
Grade 6 3 1 3 1 
Grade 7 3 1 3 1 
Grade 8 3 1 3 1 

Grade 9 3 1 

Grade 10 3 1 

Grade 11 3 1 
Algebra | 3 1 
Geometry 3 1 
Algebra Il 3 1 
Integrated Mathematics | 1 1 
Integrated Mathematics II 1 1 
Integrated Mathematics III 1 1 


*Grades 3-11 ELA/L and Integrated Mathematics | have two operational accommodated forms and 
mathematics grades 3-8 and the high school traditional assessments have three accommodated forms. 
The forms are accommodated to support Braille, large print, human reader/human signers, assistive 
technology, text-to-speech, closed captioning, and Spanish. Human reader/human signers and Spanish 
are provided for mathematics assessments only. Closed captioning is provided for ELA/L assessments 
only. 


Test Construction Activities 

After the data review meetings and prior to the test construction meetings, Pearson assessment 
specialists constructed initial versions of all the core forms, as depicted in Table 2.1. The construction 
model varied slightly between the two subject areas. 


For ELA/L, content specialists constructed the initial core forms shown in Table 2.1 based on the support 
documents and specific processes to achieve fair parallel forms. The following steps were used to 
construct the operational core ELA/L form inputs taken to the Test Construction Committee for review. 


1. constructed the online forms to match blueprint and test construction specifications 

2. constructed the paper forms to match the blueprint and test construction specifications 

3. identified accommodated and accessibility forms by evaluating the constructed forms for 
eligibility 


The ELA/L construction process included iterative steps between content specialists and 
psychometricians. Custom PARCC test construction reports (i.e., SAS reports) generated by the Pearson 
psychometric team provided information on adherence to blueprint and statistical 
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averages/distributions of item difficulty and discrimination describing the forms and allowing 
comparison of the forms. These reports facilitated content changes to better achieve the test 
construction goals. 


For mathematics, Pearson employed the use of an automatic test assembler (ATA) to select the items 
for the initial forms. Based on the blueprints and other test construction goals and specifications, the 
ATA was able to create sets of items best satisfying the statistical parameters outlined in the test 
construction specifications; however, the ATA was unable to sequence the items as required by the 
PARCC Mathematics Sequencing Guidelines. Sequencing was conducted by assessment specialists who 
ordered the items according to the sequencing guidelines. To achieve the appropriate linking design, 
assessment specialists created linking item sets from an ATA-generated linking blueprint; these sets are 
shared across forms using the strategy described in Section 2.2.4. The following steps were used to 
construct the linking sets and operational core form inputs taken to the Test Construction Committee 


for review. 
1. pulled linking online blueprint (ATA) 
2. sequenced blueprint and created linking item sets 
3. pulled online forms using linking sets (ATA) 
4. constructed the online forms 
5. constructed the paper forms 
6. identified accommodated and accessibility forms 


Similar to the ELA/L construction process, mathematics included iterative steps between assessment 
specialists and psychometricians. Custom PARCC test construction reports (i.e., SAS reports) generated 
by the Pearson psychometric team provided information on adherence to blueprint and statistical 
averages/distributions of item difficulty and discrimination allowing a comparison of the forms and 
facilitating content changes to better achieve the test construction goals. Since the mathematics forms 
were generated by the ATA, psychometricians could also generate the SAS reports prior to content 
experts reviewing the forms. 


Pearson assessment specialists identified forms for each grade/subject suitable for use as the 
accommodated forms. The content of these forms was also reviewed by Pearson accessibility specialists 
allowing for content changes prior to the Test Construction Committee meetings. 


These test construction activities provided significant inputs to commence the meetings including: 
e the proposed items for the initial operational core forms and the accommodated forms 
described above 


e reports describing each form and comparing parallel forms 
e recommended accommodated forms 
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Test Construction Meeting to Review Test Construction Inputs 

Members of the State Content Item Review Committees and the Accessibility, Accommodations, and 
Fairness (AAF) OWG participated in the building of operational core forms that met PARCC summative 
assessments requirements. In that process, they met in an in-person meeting to review and make 
recommendations for changes so that test forms conformed to both the content and psychometric 
requirements of the assessment. 


Accommodated Form Review Process 

In addition to participating in many of the development activities including the State Text Review and 
the State Bias and Sensitivity Review meetings, the AAF OWG reviewed the proposed accommodated 
forms at the Test Construction Committee meeting for accessibility to make sure that the content can be 
accommodated for students with disabilities and English learners without changing the underlying 
measured construct. 


Forms were identified to support the following accommodations: 


Accommodated Base 1 
e Spanish paper (also serves Spanish LP, Spanish human reader paper) 
e Spanish human reader/human signer online 
e base accommodated paper (serves Braille, LP, human reader paper) 
e human reader/human signer online 
e assistive technology screen reader 
e assistive technology non-screen reader 
e = =ASL 


Accommodated Base 2 

e closed captioning 

e text-to-speech first form 

e Spanish online 

e Spanish text-to-speech 
Accommodated Base 3 

e text-to-speech second form 


All Human Reader/Human Signers are mathematics only. ELA/L Human Reader uses a main online form. 
Spanish is mathematics only. Closed captioning is ELA/L only. 


At the conclusion of the meetings, all test forms were constructed to meet test blueprints and PARCC 
requirements, and to the extent possible, reflect the operational linking design. Each test form reflected 
the test blueprint in terms of content, item types, and test length, as well as expected difficulty and 
performance along the ability continuum. Linking sets were proportionally representative of the 
operational test blueprint. The operational core forms, linking set forms, and field-test forms were 
reviewed by PARCC Forms Review Committees and approved prior to the test administration. 
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Spanish-Language Assessments for Mathematics 

For English learners, PARCC offers the mathematics assessments in Spanish, as well as large print and 
text-to-speech (TTS) versions of the test in Spanish. Once the operational form was approved, the form 
was sent to Pearson’s subcontractor, Teneo, for transadaption of the items. Transadaption differs from 
translation in that it takes into consideration the grade-level appropriateness of the words, as well as 
the linguistic and cultural differences that exist between speakers of two different languages. 
Accounting for these differences allows the item to measure the achievement of Spanish language 
speakers in the same way that the original version of the item does for native speakers of English. The 
PARCC Spanish Glossary provided guidance to the translator conducting the transadaption in grade-level 
and culturally appropriate ways of transadapting the items. For the Spanish language TTS form, the 
alternate text (used for description and/or text in art and graphics) was transadapted from the alternate 
text for the English language version of the TTS form. Phonetic mark-up, which guides how the TTS 
reader pronounces content-specific words and phrases, was also applied in this process. 


In addition to the expert review of potential content for all accommodated forms conducted by the AAF 
OWG with assistance from content experts at the test construction meetings, the transadapted forms 
underwent three additional quality checks: a Pearson Spanish copy edit services review and approval, an 
AAF OWG review and approval, and a Spanish DIF analysis after the administration, when possible. 


2.2.4 Linking Design of the Operational Test 


This section begins with a discussion of special considerations for selection of linking items, followed by 
two examples of the graphical representations of the linking designs for ELA/L and for mathematics. To 
support the goal of score comparability within and across administrations and years, PARCC 
implemented a hybrid approach that incorporated the strengths of common item linking and randomly 
equivalent groups. The use of repeated operational core items was leveraged for common item linking. 
In addition, all forms were available throughout the operational administration, with spiraling at the 
student level, leveraged to support linking through randomly equivalent groups. 


The PARCC operational test forms involved various types of linking: horizontal linking, testing mode 
linking, and across-administration linking. Horizontal linking consisted of linking items, or common 
items, included in multiple forms in a single administration. The horizontal linking was achieved through 
a daisy-chain strategy. This strategy links multiple operational forms together in a ring, where each 
operational form shares some items with a preceding form and some items with a following form, and 
the last form also shares some items with the first form. Together, all the shared items make up the 
horizontal linking set. All forms for the grade and subject are connected, but not identical (e.g., Ais 
connected to B, B is connected to C, and C is connected to A). Testing mode linking consisted of common 
items placed in computer-based forms and paper-based forms within an administration to support the 
development of scores on the same reporting scale. Across-administration linking, or year-to-year 
linking, consisted of common items included in two different administrations. The placement of linking 
items across forms or administrations supports the development of comparable scores. 
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Linking item sets can be internal or external linking sets. Internal linking sets consist of common items in 
operational positions such that the items contribute to the students’ scores. External linking sets consist 
of common items in positions resulting in the items not contributing to students’ scores. The current 
linking designs included both external and internal linking sets. The horizontal linking across forms 
within an administration and across-administration linking included internal linking sets. The testing 
mode linking included both internal and external linking sets. 


For ELA/L, the horizontal linking designs for the online test forms were based on the number of unique 
test forms constructed for a grade. After constructing the unique test forms, the test forms were divided 
into sections and sections were dispersed across additional forms such that each section appeared on 
two forms. As a result, the operational linking sets represented full test blueprints. This means that 
linking items were selected to reflect the content balance, task models, types of items, and cognitive 
complexity of the full PARCC assessment. 


For mathematics, the ATA pulled an initial blueprint linking set that was divided into item sets and 
distributed across the online forms following a daisy-chain strategy, as depicted in Figure 2.2. 


The paper forms for both subjects were generated from the online forms. In response to several 
practical constraints based on the number of forms constructed for each mode and to meet the 
blueprints (e.g., inclusion of technology-enhanced items in CBT forms), there was no one online form 
that was administered intact in the paper delivery mode at any grade level. For example, technology- 
enhanced items from online forms were replaced in the paper forms with items from similar content, 
but appropriate for PBT. However, for both subjects, the content on paper forms overlaps with that on 
the online forms. 


2.2.5 Graphical Representation of PARCC Operational Test Linking Design 


This section includes two examples of graphical representations to illustrate the horizontal linking 
designs described above. Designs for across-administrations linking are not included in these graphs. 
Note that to the extent possible, item set (a) for within-year equating and item set (b) for across-year 
equating will be the same set of items. Limitations in achieving identical linking sets include the release 
of some of last year’s content and exposure concerns. 


Figure 2.1 illustrates the linking design for ELA/L for grades 6-11 CBT forms (01, O2, 03) and PBT forms 
(P1, P2). Each form was constructed to include two linking sets, one text and one task, accounting for 
approximately 33 percent of the score points. The set of forms for a grade/subject linked through a 
daisy-chain model such that each form was linked to two adjacent forms. Additional new content had 
exposure on multiple forms, providing stronger within-year linking. Grades 3-5 linking design was 


similar. 
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PARCC ELA/L Grades 6-11 
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Figure 2.1 PARCC ELA/L CBT and PBT Linking Design (Grades 6-11) 


Figure 2.2 illustrates the linking design for mathematics for grades 3—5 CBT forms (O01, 02, O3) and PBT 
forms (P1, P2). Each form was constructed to include three linking sets, accounting for approximately 38 
percent of the score points. Linking sets were positioned in different units. The set of forms for a 
grade/subject linked through a daisy-chain model such that each form was linked to two adjacent forms. 
Additional new content had exposure on multiple forms, providing stronger within-year linking. Grades 
6-8 and high school linking designs were similar. 
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PARCC Mathematics Grades 3-5 
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Figure 2.2 PARCC Mathematics CBT and PBT Linking Design (Grades 3-5) 


2.2.6 Field Test Data Collection Overview 


Field-test items were embedded in the spring operational forms to collect data for psychometric analysis 
necessary to support the assessment system for future administrations. Field-test administration 
entailed paper and computer administration modes, with computer administration as the dominant 
mode. The ELA/L embedded field-test items were administered to a sample of students. 


Field-test sets were constructed to balance the expected cognitive load and difficulty across forms, 
reflected in the number of points, distribution of task types, and balance of passages for ELA/L. Forms 
for each content area were spiraled at the student level. The data collection design entailed two 
conditions. Condition 1, which comprised the mathematics assessment, was an embedded census field- 
test model in which all students taking the summative assessment participated in the field test. 


Under Condition 2, which comprised the ELA/L assessment, PARCC sampled approximately one-third of 
the schools across the consortium states. Students in the sampled schools took forms containing ELA/L 
embedded field-test tasks. Schools were selected so that the sample for each ELA/L assessment was 
representative of the general PARCC testing populations in terms of achievement (i.e., average scale 
score and percentage of student at Level 4 and Level 5 in the previous year) and demographics (i.e., 
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ethnicity composition, percentage of economically disadvantaged, English learners, and students with 
disabilities). A three-year sampling plan was created such that if a given school was part of the ELA/L 
field test one year (e.g., Spring 2016), it would not be required to participate in the field test for the 
subsequent two years (e.g., Spring 2017 and Spring 2018). 
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Section 3: Test Administration 


3.1 Testing Windows 


The 2016-2017 operational administration of PARCC assessments included a fall/winter block 
administration beginning in the fall of 2016 as well as a spring administration in the spring of 2017. Each 
PARCC assessment consisted of multiple units, and additionally, one of the mathematics units for grade 
7 and high school course assessments was split into two sections: a non-calculator section and a 
calculator section. 


Table 3.1 PARCC Fall/Winter Block 2016 and Spring 2017 Testing Windows 


Fall/Winter Block 2016 November 14, 2016- 
January 27, 2017 


March 13, 2017- 


Spring 2017 June 9, 2017 


3.2 Test Security and Administration Policies 


The administration of any PARCC assessment is a secure testing event. Maintaining the security of test 
materials before, during, and after the test administration is crucial to obtaining valid and reliable 
results. School Test Coordinators are responsible for ensuring that all personnel with authorized access 
to secure materials are trained in and subsequently act in accordance with all security requirements. 


School Test Coordinators must implement chain-of-custody requirements for specified materials. School 
Test Coordinators are responsible for distributing materials to Test Administrators, collecting materials 
from Test Administrators, returning secure test materials, and securely destroying certain specified 
materials after testing. 


The administration of the PARCC assessment includes both secure and non-secure materials, and these 
materials are further delineated by whether they are “scorable” or “nonscorable,” depending on 
whether the assessments were administered via paper/pencil (i.e., paper-based assessments) or online 
(i.e., Computer-based assessments). For the 2016-2017 paper-based administration, students used 
paper-based answer documents (except in grade 3 where students responded directly into test 
booklets). About 96 percent of the PARCC assessments administered during the 2016-2017 
administration were online assessments, and about 4 percent were paper-based assessments. 


Secure vs. Non-Secure Materials 
PARCC defines secure materials as those that must be closely monitored and tracked to prevent 
unauthorized access to or prohibited use or distribution of secure content such as test items, reading 
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passages, student work, etc. For paper-based tests, secure materials include both used and unused test 
booklets and used scratch paper, while for computer-based tests, secure materials include student 
testing tickets, secure administration scripts (e.g., mathematics read-aloud), and used scratch paper. 
PARCC defines non-secure materials as any authorized testing materials that do not include secure 
content (e.g., test items or student work). These include test administration manuals, unused scratch 
paper, and mathematics reference sheets that have not been written upon, etc. 


Scorable vs. Nonscorable Materials 

Paper-based assessments have both scorable and nonscorable materials while computer-based 
assessments have only nonscorable materials. Scorable materials for paper-based assessments consist 
of used (includes student work) test booklets (grade 3) and answer documents (grades 4 and above) 
only. Scorable materials must be returned to the vendor to be scored. All other materials for paper- 
based testing, such as blank (i.e., unused) test booklets, test administration manuals, scratch paper, 
mathematics reference sheets, etc., are deemed nonscorable. For computer-based tests, there are no 
scorable materials as student work is submitted electronically for scoring; thus there are limited physical 
materials to return (e.g., secure administration scripts for certain accommodations). 


Students taking the computer-based test may not have access to secure test materials before testing, 
including printed student testing tickets. Printed mathematics reference sheets (if applicable) and scratch 
paper must be new and unmarked. 


Students taking the paper-based test may not have access to scorable or nonscorable secure test 
content before or after testing. Scorable secure materials that are to be provided by Test Administrators 
to students include test booklets (grade 3) or answer documents (grades 4 through high school). 
Nonscorable secure materials that are distributed by Test Administrators to paper-based testing 
students include large print test booklets, braille test booklets, scratch paper (paper used by students to 
take notes and work through items), and printed mathematics reference sheets (grades 5 through 8 and 
high school). 


School Test Coordinators are required to maintain a tracking log to account for collection and 
destruction of test materials, including mathematics reference sheets and scratch paper written on by 
students. As part of the test administration policy, schools are required to maintain the Chain-of- 
Custody Form or tracking log of secure materials for at least three years unless otherwise directed by 
state policy. Copies of the Chain-of-Custody Form for paper-based testing are included in each Local 
Education Agency (LEA) or school’s test materials shipment. 


Test Administrators are not to have extended access to test materials before or after administration 
(except for certain accessibility or accommodations purposes). Test Administrators must document the 
receipt and return of all secure test materials (used and unused) to the School Test Coordinator 
immediately after testing. 


All PARCC test security and administration policies are found in the PARCC Test Coordinator Manual and 
the PARCC Test Administrator Manuals. State security and administration policies may exceed that of 
the PARCC policies. State-specific policies are included in Appendix C of the Test Coordinator Manual. 
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3.3 Accessibility Features and Accommodations 
3.3.1 Participation Guidelines for PARCC Assessments 


All students, including students with disabilities and English learners, are required to participate in 
statewide assessments and have their assessment results be part of the state’s accountability systems, 
with narrow exceptions for English learners in their first year in a U.S. school, and certain students with 
disabilities who have been identified by the Individualized Education Program (IEP) team to take their 
state’s alternate assessment. All eligible students will participate in the PARCC ELA/L and mathematics 
assessments. Federal laws governing student participation in statewide assessments include the No 
Child Left Behind Act of 2001 (NCLB), the Individuals with Disabilities Education Act of 2004 (IDEA), 
Section 504 of the Rehabilitation Act of 1973 (reauthorized in 2008), and the Elementary and Secondary 
Education Act (ESEA) of 1965, as amended. All students can receive accessibility features on PARCC 
assessments. 


Four distinct groups of students may receive accommodations on PARCC assessments: 


students with disabilities who have an Individualized Education Program (IEP); 
students with a Section 504 plan who have a physical or mental impairment that substantially 
limits one or more major life activities, have a record of such an impairment, or are regarded as 
having such an impairment, but who do not qualify for special education services; 

3. students who are English learners; and 

4. students who are English learners with disabilities who have an IEP or 504 plan. These 
students are eligible for accommodations intended for both students with disabilities and 
English learners. 


Testing accommodations for students with disabilities or students who are English learners (EL) must be 
documented according to the guidelines and requirements outlined in the PARCC Accessibility Features 
and Accommodations Manual. 


3.3.2 PARCC Accessibility System 


Through a combination of universal design principles and accessibility features, PARCC has designed an 
inclusive assessment system by considering accessibility from initial design through item development, 
field testing, and implementation of the assessments for all students, including students with 
disabilities, English learners, and English learners with disabilities. Accommodations may still be needed 
for some students with disabilities and English learners to assist in demonstrating what they know and 
can do. However, the accessibility features available to students should minimize the need for 
accommodations during testing and ensure the inclusive, accessible, and fair testing of the diverse 
students being assessed. 


3.3.3 What are Accessibility Features? 
On the PARCC computer-based assessments, accessibility features are tools or preferences that are 


either built into the assessment system or provided externally by Test Administrators, and may be used 
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by any student taking the PARCC assessments (i.e., students with and without disabilities, gifted 
students, English learners, and English learners with disabilities). Since accessibility features are 
intended for all students, they are not classified as accommodations. Students should have the 
opportunity to select and practice using them prior to testing to determine which are appropriate for 
use on the PARCC assessment. Consideration should be given to the supports a student finds helpful and 
consistently uses during instruction. Practice tests that include accessibility features are available for 
teacher and student use throughout the year. Practice tests are available at parcc.pearson.com. 


3.3.4 Accommodations for Students with Disabilities and English Learners 


It is important to ensure that performance in the classroom and on assessments is influenced minimally, 
if at all, by a student’s disability or linguistic/cultural characteristics that may be unrelated to the 
content being assessed. For PARCC assessments, accommodations are considered to be adjustments to 
the testing conditions, test format, or test administration that provide equitable access during 
assessments for students with disabilities and students who are English learners. In general, the 
administration of the assessment should not be the first occasion on which an accommodation is 
introduced to the student. To the extent possible, accommodations should: 


e provide equitable access during instruction and assessments; 
e mitigate the effects of a student’s disability; 

e not reduce learning or performance expectations; 

e not change the construct being assessed; and 

e not compromise the integrity or validity of the assessment. 


Accommodations are intended to reduce and/or eliminate the effects of a student’s disability and/or 
English language proficiency level; however, accommodations should never reduce learning 
expectations by reducing the scope, complexity, or rigor of an assessment. Moreover, 
accommodations provided to a student on the PARCC assessments must be generally consistent with 
those provided for classroom instruction and classroom assessments. There are some accommodations 
that may be used for instruction and for formative assessments that are not allowed for the summative 
assessment because they impact the validity of the assessment results—for example, allowing a student 
to use a thesaurus or access the Internet during a PARCC assessment. There may be consequences (e.g., 
excluding a student’s test score) for the use of non-allowable accommodations during PARCC 
assessments. It is important for educators to become familiar with PARCC policies regarding 
accommodations used for assessments. 


To the extent possible, accommodations should adhere to the following principles. 


e Accommodations enable students to participate more fully and fairly in instruction and 
assessments and to demonstrate their knowledge and skills. 

e Accommodations should be based upon an individual student’s needs rather than on the 
category of a student’s disability, level of English language proficiency alone, level of or access to 
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grade-level instruction, amount of time spent in a general classroom, current program setting, 
or availability of staff. 

e Accommodations should be based on a documented need in the instruction/assessment setting 
and should not be provided for the purpose of giving the student an enhancement that could be 
viewed as an unfair advantage. 

e Accommodations for students with disabilities must be described and documented in the 
student’s appropriate plan (i.e., either a 504 plan or an approved IEP), and must be provided if 
they are listed. 

e Accommodations for English learners should be described and documented. 

e Students who are English learners with disabilities are eligible to receive accommodations for 
both students with disabilities and English learners. 

e Accommodations should become part of the student’s program of daily instruction as soon as 
possible after completion and approval of the appropriate plan. 

e Accommodations should not be introduced for the first time during the testing of a student. 

e Accommodations should be monitored for effectiveness. 

e Accommodations used for instruction should also be used, if allowable, on local district 
assessments and state assessments. 


In the following scenarios, the school must follow each state’s policies and procedures for notifying the 
state assessment office: 


e astudent was provided a test accommodation that was not listed in his or her IEP/504 
plan/documentation for an English learner, or 

e astudent was not provided a test accommodation that was listed in his or her IEP/504 
plan/documentation for an English learner. 


3.3.5 Unique Accommodations 


PARCC provides a comprehensive list of accessibility features and accommodations in the PARCC 
Accessibility Features and Accommodations Manual that are designed to increase access to PARCC 
assessments and that will result in valid, comparable assessment scores. However, students with 
disabilities or English learners may require additional accommodations that are not already listed. 
PARCC states individually review requests for unique accommodations in their respective states and 
provide a determination as to whether the accommodation would result in a valid score for the student, 
and if so, would approve the request. 


3.3.6 Emergency Accommodations 


An emergency accommodation may be appropriate for a student who incurs a temporary disabling 
condition that interferes with test performance shortly before or during the PARCC assessment window. 
A student, whether or not they already have an IEP or 504 plan, may require an accommodation as a 
result of a recently occurring accident or illness. Cases include a student who has a recently fractured 
limb (e.g., arm, wrist, or shoulder); a student whose only pair of eyeglasses has broken; or a student 
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returning to school after a serious or prolonged illness or injury. An emergency accommodation should 
be given only if the accommodation will result in a valid score for the student (i.e., does not change the 
construct being measured by the test[s]). If the principal (or designee) determines that a student 
requires an emergency accommodation on the PARCC assessment, an Emergency Accommodation Form 
must be completed and maintained in the student’s assessment file. If required by a PARCC state, the 
school may need to consult with the state or district assessment office for approval. The parent must be 
notified that an emergency accommodation was provided. If appropriate, the Emergency 
Accommodation Form may also be submitted to the District Assessment Coordinator to be retained in 
the student’s central office file. Requests for emergency accommodations will be approved after it is 
determined that use of the accommodation would result in a valid score for the student. 


3.3.7 Student Refusal Form 


If a student refuses an accommodation listed in his or her IEP, 504 plan, or if required by the PARCC 
member state, an English learner plan, the school should document in writing that the student refused 
the accommodation, and the accommodation must be offered and remain available to the student 
during testing. This form must be completed and placed in the student’s file and a copy must be sent to 
the parent on the day of refusal. Principals (or designee) should work with Test Administrators to 
determine who, if any others, should be informed when a student refuses an accommodation 
documented in an IEP, 504 plan, or (if required by the PARCC member state) English learner plan. 


3.4 Testing Irregularities and Security Breaches 


Any action that compromises test security or score validity is prohibited. These may be classified as 
testing irregularities or security breaches. Below are examples of activities that compromise test 
security or score validity (note that these lists are not exhaustive). It is highly recommended that School 
Test Coordinators discuss other possible testing irregularities and security breaches with Test 
Administrators during training. 


Examples of test security breaches and irregularities include but are not limited to: 


e Electronic Devices 
o Using a cell phone or other prohibited handheld electronic device (e.g., smartphone, 
iPod, smart watch, personal scanner) while secure test materials are still distributed, 
while students are testing, after a student turns in his or her test materials, or during a 
break 
=" Exception: Test Coordinators, Technology Coordinators, Test Administrators, 
and Proctors are permitted to use cell phones in the testing environment only in 
cases of emergencies or when timely administration assistance is needed. LEAs 
may set additional restrictions on allowable devices as needed. 
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Test Supervision 


[e) 


oO 0 0 0 


(2) 


Coaching students during testing, including giving students verbal or nonverbal cues, 
hints, suggestions, or paraphrasing or defining any part of the test 

Engaging in activities (e.g., grading papers, reading a book, newspaper, or magazine) 
that prevent proper student supervision at all times while secure test materials are still 
distributed or while students are testing 

Leaving students unattended for any period of time while secure test materials are still 
distributed or while students are testing 

Deviating from testing time procedures 

Allowing cheating of any kind 

Providing unauthorized persons with access to secure materials 

Unlocking a test in PearsonAccess"™™ during non-testing times 

Failing to provide a student with a documented accommodation or providing a student 
with an accommodation that is not documented and therefore is not appropriate 
Allowing students to test before or after the state’s test administration window 


Test Materials 


oO 0 0 


Losing a student test booklet or answer document 
Losing a student testing ticket 
Leaving test materials unattended or failing to keep test materials secure at all times 
Reading or viewing the passages or test items before, during, or after testing 
= Exception: Administration of a human reader/signer accessibility feature for 
mathematics or accommodation for English language arts/literacy, which 
requires a Test Administrator to access passages or test items 
Copying or reproducing (e.g., taking a picture of) any part of the passages or test items 
or any secure test materials or online test forms 
Revealing or discussing passages or test items with anyone, including students and 
school staff, through verbal exchange, email, social media, or any other form of 
communication 
Removing secure test materials from the school’s campus or removing them from locked 
storage for any purpose other than administering the test 


Testing Environment 


Oo 


[e) 


Allowing unauthorized visitors in the testing environment 


Failing to follow administration directions exactly as specified in the Test Administrator 
Manual 

Displaying testing aids in the testing environment (e.g., a bulletin board containing 
relevant instructional materials) during testing 


All instances of security breaches and testing irregularities must be reported to the School Test 


Coordinator immediately. The Form to Report a Testing Irregularity or Security Breach must be 


completed within two school days of the incident. 
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If any situation occurred that could cause any part of the test administration to be compromised, 
schools should refer to the PARCC Test Coordinator Manual for each state’s policy and immediately 
follow those steps. Instructions for the School Test Coordinator or LEA Test Coordinator to report a 
testing irregularity or security breach is available in the PARCC Test Coordinator Manual. 


3.5 Data Forensics Analyses 


Maintaining the validity of test scores is essential in any high-stakes assessment program, and 
misconduct represents a serious threat to test score validity. When used appropriately, data forensic 
analyses can serve as an integral component of a wider test security protocol. The results of these data 
forensic analyses may be instrumental in identifying potential cases of misconduct for further follow-up 
and investigation. 


In 2016-2017, PARCC conducted the following data forensics analyses on its operational assessments: 


e Response Change Analysis 

e Aberrant Response Analysis 

e Plagiarism Analysis 

e Longitudinal Performance Modeling 
e Internet and Social Media Monitoring 
e Off-Hours Testing Monitoring 


An overview of each data forensics analysis method is provided next. 
3.5.1 Response Change Analysis 


Response change analysis looks at how often student answers are changed, focusing specifically on an 
excessive number of wrong answers changed to right answers. In traditional paper-based, multiple- 
choice testing programs, this is sometimes referred to as “erasure analysis”*. The rationale for erasure 
analysis is that a teacher or administrator who is intent on improving classroom performance might be 
motivated to change student responses after the answer sheets are collected. A clustered number of 
student answer documents from the same school or classroom with unusually high numbers of answers 
changed from wrong to right might provide evidence to support follow-up investigation. PARCC’s 
response change analysis extended the traditional erasure method to account for issues specific to 
computer-based testing as well as the variety of item types on the PARCC assessments, such as partial- 
credit, multi-part, and multiple-select items. 


3 The term “erasure analysis” is sometimes objected to because it is inferential rather than descriptive. A more 
descriptive term is “mark discrimination analysis,” which recognizes that the scanning approach makes 
discriminations among the darkness of selected answer choices when multiple responses to a multiple-choice item 
are detected during answer sheet processing. 
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3.5.2 Aberrant Response Analysis 


Aberrant response pattern detection analysis looks at the unusualness of student responses compared 
with what would be expected. Most simply, this can be thought of as quantifying the extent to which 
higher-scoring students miss easy questions and lower-scoring students answer difficult questions 
correctly. While it would be difficult to draw a definitive inference about a single student flagged as 
having an aberrant response pattern, a cluster of students with aberrant response patterns within a 
classroom or school might warrant further investigation. 


3.5.3 Plagiarism Analysis 


Plagiarism analysis compares the responses given for a group of written composition items, looking for 
high degrees of similarity. For the PARCC assessments, the primary item type of interest was the prose 
constructed-response (PCR) tasks in the English Language Arts and Literacy (ELA/L) content area. This 
analysis was conducted for PCR tasks administered online using some of the same artificial intelligence 
(Al) techniques that are applied in automated essay scoring. Specifically, this method was based on 
Latent Semantic Analysis (LSA) technology to detect possible plagiarism. Using LSA, the content of each 
constructed response was compared against the content of every other constructed response and a 
measure that indicated the degrees of similarity was generated for each pair of response comparison. 
Because LSA provided a semantic representation of language, rather than a syntactic or word-based 
representation, it allowed the detection of potential copying behaviors, even when test takers or 
administrators substituted synonymous words or phrases. 


3.5.4 Longitudinal Performance Monitoring 


Longitudinal performance modeling evaluates the performance on PARCC assessments across test 
administrations and identifies unusual performance gains in the unit of interest (e.g., school or district). 
The original proposal was to use the cumulative logit regression (CLR) model approach (Clark, Skorupski, 
Jirka, McBride, Wang & Murphy, 2014) to identify unusual changes in test performance across two 
consecutive administrations of the PARCC assessment. A research study comparing the cumulative logit 
regression model and the weighted least squares (WLS) regression approach found that the cumulative 
logit regression model worked well when there were more units exhibiting unexpected performance. 
WLS resulted in lower false positives when few units exhibited unexpected performance. In addition, the 
units flagged by WLS were almost always flagged by CLR. 


At the November 2016 Technical Advisory Committee (TAC) meeting, the results from this study were 
discussed and the recommendation to use the Weighted Least Squares methodology was endorsed by 
the TAC. In January of 2017, the State Leads approved the TAC recommendation. 


In the weighted least squares regression approach, mean current year scale scores are regressed on 
mean prior year scale scores, weighting by unit sample size. The model is fit using SAS PROC REG or the 
Im function in R. Standardized residuals are calculated by dividing raw residuals by their respective 
standard deviations. Units with a standardized residual exceeding 3.0 are flagged for unexpected 
performance. 
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3.5.5 Internet and Social Media Monitoring 


Internet and social media monitoring was conducted by Caveon, LLC. Caveon’s team monitored English- 
language websites and searchable forums that were publicly available for suspected proxy testing 
solicitations and website postings that contain, or appear to contain, infringements of PARCC’s 
protected operational test content. The Internet and social media outlets monitored included popular 
websites (such as Facebook and Twitter), blogs, discussion forums, video archives, document archives, 
brain dumps, auction sites, media outlets, peer-to-peer servers, etc. Caveon’s process generated regular 
updates that categorize identified threats by level of actual or potential risk based upon the 
representations made on the websites, or actual analysis of the proffered content. For example, 
categorizations typically ranged from “cleared” (lowest risk but bookmarked for continued monitoring) 
to “severe” (highest risk). Note that this process only considered potential breaches of secure item 
content, not violations of testing administration policies. Potential breaches were reported directly to 
the state(s) implicated for further action. Summary reports describing the threats were provided to 
PARCC through notification emails. 


3.5.6 Off-Hours Testing Monitoring 


Off-hours testing monitoring checks for suspicious testing activities at test administration locations 
occurring outside of the set windows for computer-based testing sessions. PARCC states established set 
start and end times for administering computer-based assessments. Based on these hours, authorized 
users (that is, users with the State Role) were allowed to override the start and end times for a test 
session. The off-hours testing monitoring process tracked such occurrences and logged them in an 
operational report, which listed the sessions within an organization that selected to test outside the set 
window. PARCC states could use this report to follow-up with the organizations identified in the report. 
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Section 4: Item Scoring 


4.1 Machine-Scored Items 
4.1.1 Key-Based Items 


Pearson performed a key review prior to the test administration to verify that the scoring (answer) keys 
were correct for each item. Once the forms were constructed and approved by PARCC for publication, 
an independent key review was performed by an experienced third-party vendor. The vendor reviewed 
each item and confirmed that the key was correct. If discrepancies were identified, a Pearson senior 
content specialist or content manager reviewed the flagged item(s) and worked with the item 
developers to resolve the issue. 


4.1.2 Rule-Based Items 


Rule-based scoring refers to item types that use various scoring models. PARCC uses Question and Test 
Interoperability (QTI) item type implementation based on scoring model rules. Examples of these item 
types include “choice interaction,” which presents a set of choices where one or more choices can be 
selected; text entry, where the response is entered in a text box; hot spot or text interaction, where an 
area in a graph or text in a paragraph (for example) can be highlighted; or match interaction, where an 
association can be made between pairs of choices in a set. These items include the scoring rules and 
correct responses as part of their item XML (markup language) coding. 


During the initial stages of item development for PARCC, Pearson staff worked closely with PARCC to 
first delineate the rules for the scoring rubrics and then to adjust those rules based on student 
responses. During the item tryout’ planning phase, Pearson content staff received input from PARCC 
staff to develop a thorough rule-based scoring process that met PARCC needs. 


Pearson worked with the item developers to review initial scoring rules created during the item 
development. Once the rule-based scoring process was approved by PARCC, and prior to test 
construction, Pearson content staff worked closely with the item developers to finalize scoring rubrics 
for items to be scored via the rule-based scoring method. The proposed scoring rubrics were sent to 
PARCC for review, and if any additional changes were needed or new rules added, Pearson documented 
and applied the requested edits. 


During test construction, Pearson monitored and evaluated the scoring and updated the scoring keys/ 
scoring rules in the item bank. After the tryout items were scored, Pearson prepared a frequency 
distribution of student responses for each item or task scored using a rule-based approach and 
compared this to the expected response based on correct answers to ensure that scoring keys and rules 
were appropriately applied. The content team does this by analyzing the student response data to 
determine if scoring is acceptable using the item metadata and the student response file in conjunction 
with any potential item issues as flagged by psychometrics. These frequency distributions included an 


4 The item tryout was a set of item studies conducted in spring 2015. 
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indication of right/wrong and other identifying information defined by PARCC and those items that 
showed a statistical anomaly, whereby the frequency distribution was outside of the expected range, 
were sent to content experts to verify that the items were coded with the correct key. 


Following the Rule-Based Scoring Educator Committee’s review, which occurred prior to year one test 
construction, Pearson analyzed the feedback from the committees and made recommendations about 
adjustments to the scoring rubrics based on the results of the reviews. Upon submission of the results, 
Pearson worked with PARCC staff to discuss these findings and determine next steps prior to the 
completion of scoring. In subsequent years as scoring inquiries arise throughout the process of test 
construction, forms creation, testing, scoring, and psychometric analysis, items with scoring 
discrepancies are brought before the PARCC Priority Alert Task Force for resolution. This committee 
consists of representatives from each state as well as PARCC and Pearson content specialists. 


Following the initial development of the PARCC rule-based scoring rubrics, Pearson has continued to 
monitor and evaluate new item development to ensure the scoring rules established are maintained 
within all item types as approved. 


Pearson continues to use several avenues to monitor scoring each year. Prior to testing, a third-party 
key review checks operational and field test items for correct keys. Any disputed items go to a second 
review with Pearson content experts and anything still in question is taken before the PARCC task force 
for review and possible key change. During testing, Pearson creates early testing files for frequency 
distribution analysis whereby items where an incorrect key receives a high distribution of responses are 
further evaluated for accuracy. After testing, all responses are again evaluated for distribution of 
responses and potential scoring abnormalities during psychometric analysis. Any change in scoring that 
may be requested as a result of the psychometric analysis is also taken before the PARCC task force for 
decisions. These processes are the same for both paper and online modes of testing. 


4.2 Human or Hand-scored Items 


PARCC constructed-response items were scored by human scorers in a process referred to as hand- 
scoring. Online training units were used to train all scorers. The online training units included prompts 
(items), passages, rubrics, training sets, and qualification sets. Scorers who successfully completed the 
training and qualified, demonstrating they could correctly score student responses based on the 
guidelines in the online training units, were permitted to score student responses using the ePEN2 
(Electronic Performance Evaluation Network, second generation) scoring platform. All online and paper 
responses were scored within the ePEN2 system. Pearson monitored quality throughout scoring. 


Pearson staff roles and responsibilities were as follows: 


e Scorers applied scores to student responses. 

e Scoring Supervisors monitored the work of a team of scorers through review of scorer statistics 
and backreading, which is a review of responses scored by each scorer. When backreading, a 
supervisor sees the scores applied by scorers, which helps the supervisor provide additional 
coaching or instruction to the scorer being backread. 
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e Scoring Directors managed the scoring quality of a subset of items and monitored the work of 
supervisors and scorers for their assigned items. Directors backread responses scored by 
supervisors and scorers as part of their quality-monitoring duties. 

e English language arts/literacy (ELA/L) and mathematics content specialists managed the scoring 
quality and monitored the work of the Scoring Directors. 

e Project Managers documented the procedures, identified risks, and managed day-to-day 
administrative matters. 

e A Program Manager provided oversight for the entire scoring process. 


All Pearson employees involved in the scoring or the supervision of scoring possessed at least a four- 
year college degree. 


4.2.1 Scorer Training 


Key steps in the development of scorer training materials were rangefinding and rangefinder review 
meetings where educators and administrators from PARCC states met to interpret the scoring rubrics 
and determine consensus scores for student responses. Rangefinding meetings were held prior to 
scoring field-test items, and rangefinder review meetings were held prior to scoring operational items. 


At rangefinding meetings, educators and administrators from PARCC states reviewed student responses 
and used scoring rubrics to determine consensus scores. Those responses scored in rangefinding were 
used to create field-test scorer training sets. After items were selected for operational testing, PARCC 
educators and administrators attended rangefinder review meetings to review and approve proposed 
operational training sets. 


When developing scorer training materials, Pearson Scoring Directors carefully reviewed detailed notes 
and records from PARCC rangefinding and rangefinder review committee meetings. Training sets were 
developed using the responses scored by the committees and additional suitable student response 
samples (as needed). PARCC reviewers reviewed and approved all scorer training sets prior to scorer 
training. 


During training, scorers reviewed training sets of scored student responses with annotations that 
explained the rationale for the score assigned. The anchor set was the primary reference for scorers as 
they internalized the rubric during training. Each anchor set consisted of responses that were clear 
examples of student performance at each score point. The responses selected were representative of 
typical approaches to the task and arranged to reflect a continuum of performance. All scorers had 
access to the anchor set when they were training and scoring and were directed to refer to it regularly 
during scoring. 


Practice sets were used in training to help trainees practice applying the scoring guidelines. Scorers 
reviewed the anchor sets, scored the practice sets, and then were able to compare their assigned scores 
for the practice sets to the actual PARCC-assigned scores to help them learn. 
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Qualification sets were used to confirm that scorers understood how to score student responses 
accurately. Qualification sets were composed of responses that were clear examples of score points. 
Scorers were required to meet specified agreement percentages on qualification sets in order to score 
student responses. 


Pearson developed two types of training sets to train scorers: prototype and abbreviated sets. Prototype 
training sets were complete training sets consisting of anchor, practice, and qualification sets (refer to 
4.2.2 for information on the qualification process). In ELA/L, there was one prototype training set per 
task type (Research Simulation Task, Literary Analysis Task, and Narrative Writing Task) at each of the 
nine grade levels (grades 3 through 11). In mathematics, a prototype training set was built for a grouping 
of similar items for a total of approximately three to five prototype sets per grade level or course. 


The prototype training approach promoted consistency in scoring, as each subsequent abbreviated 
training set for the ELA/L task type or mathematics item grouping was based on the prototype. Once a 
prototype was chosen, full training materials were developed for that item, and at each grade level, 
scorers were trained to score a particular task type using the prototype training materials for that type. 


Abbreviated training sets were prepared for all items not selected for prototype training sets. The 
abbreviated training sets included an anchor set and two practice sets so scorers could internalize the 
scoring standards for these new items, which were similar to prototype items they had previously 
scored. 


Anchor and practice sets for both prototype and abbreviated items included annotations for each 
response. Annotations are formal written explanations of the score for each student response. 


The table below details the composition of the anchor sets, practice sets, and qualification sets. 
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Table 4.1 Training Materials Used During Scoring 


Training Set Development 


Description 


Specification 


Anchor Set 


The anchor set is the primary reference for 
scorers as they internalize the rubric during 
training. All scorers have access to the 
anchor set when they are training and 
scoring, and are directed to refer to it 
regularly. 


The anchor set comprises clear examples of 
student performance at each score point. 
The responses selected may be 
representative of typical approaches to the 
task or arranged to reflect a continuum of 
performance. 


The anchor set for mathematics prototype items 
comprises three annotated responses per score point. 


The anchor set for subsequent abbreviated items for 
mathematics comprise one to three annotated 
responses per score point. 


The anchor sets for ELA/L prototype items comprise 
three annotated responses per score point. Anchor sets 
for prototype items include separate complete anchor 
sets for each applicable scoring trait (Reading 
Comprehension and Written Expression and 
Conventions). 


Practice Sets 


Practice sets are used to help trainees 
develop experience in independently 
applying the scoring guide (the rubric) to 
student responses. Some of these 
responses clearly reinforce the scoring 
guidelines presented in the anchor set. 
Other responses are selected because they 
are more difficult to evaluate, fall near the 
boundary between two score categories, or 
represent unusual approaches to the task. 


The practice sets provide guidance and 
practice for trainees in defining the line 
between score categories, as well as 
applying the scoring criteria to a wider 
range of types of responses. 


The practice sets for mathematics prototype and 
abbreviated items include two to three sets of ten 
annotated responses. 


ELA/L practice sets for prototype items include two sets 
of five annotated responses and two sets of ten 
annotated responses. 


The subsequent ELA/L practice sets for abbreviated 
items include two sets of ten annotated responses. 
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Table 4.1 Training Materials Used During Scoring 


Training Set Development 


Description 


Specification 


Qualification Sets 


Qualification sets are used to confirm that 
scorer trainees understand the scoring 
criteria and are able to assign scores to 
student responses accurately. The 
responses in these sets are selected to 
reinforce the application of the scoring 
criteria illustrated in the anchor set. 


Scorer trainees must demonstrate 
acceptable performance on these sets by 
meeting a pre-determined standard for 
accuracy in order to qualify to score. 
Pearson scoring staff define and document 
qualifying standards in conjunction with 
PARCC prior to scoring. 


The qualification sets for mathematics prototype items 
include three sets of ten responses each (not 
annotated). 


The subsequent mathematics abbreviated items for 
mathematics do not include qualification sets. 


The qualification sets for ELA/L prototype items include 
three sets of ten responses each (not annotated). 


The subsequent ELA/L abbreviated items do not include 
qualification sets. 
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4.2.2 Scorer Qualification 


In order to score items, scorers were required to show that they were able to apply PARCC scoring 
methodology accurately through a qualification process. Scorers were asked to apply scores to three 
qualification sets consisting of ten responses each. ELA/L scorers applied a score for each trait on each 
response in the qualification sets. Literary Analysis and Research Simulation Tasks each had two traits: 
the Reading Comprehension and Written Expression trait and the Conventions trait. The Narrative 
Writing Task had two traits: Written Expression and Conventions. Mathematics scorers applied a score 
for each part of an item that was a constructed response. The number of constructed-response parts for 
each mathematics item ranged from one to four. Scorers were required to match the PARCC-approved 
score at a percentage agreed to by PARCC in order to qualify. 


For ELA/L qualification, scorers were required to meet the following three conditions: 


1. Onat least one of the three qualifying sets, at least 70 percent of the ratings on each of the two 
scoring traits (considered separately), must agree exactly with the PARCC-approved scores. 

2. Onat least two of the three qualifying sets, at least 70 percent of the ratings (combined across 
the three scoring traits) must agree exactly with the PARCC-approved scores. 

3. Combining over the three qualifying sets and across the two scoring traits, at least 96 percent of 
the ratings must be within one point of the PARCC-approved scores. 


For mathematics qualification, the requirements were based on the item types and score point ranges. 
Because mathematics items can have one or more scoring traits, a scorer needed to achieve the 
following requirements separately for each scoring trait (when applicable to the item): 


Table 4.2 Mathematics Qualification Requirements 


Category Score Point Range Perfect Agreement Within One Point 
2-category 0-1 90% 100% 
3-category 0-2 80% 96% 
4-category 0-3 70% 96% 
5-category 0-4 70% 95% 
7-category 0-6 70% 95% 


On at least two of the three qualifying sets, a scorer was required to meet the “perfect agreement” 
percentage indicated in the table above for each category. “Perfect agreement” was achieved when the 
scores applied exactly matched the PARCC-approved scores. Over the three qualifying sets, a scorer was 
required to meet the “within one point” percentage indicated in the table above for each category. The 
average is exclusive to each trait, so an item with multiple scoring traits would have multiple trait rating 
averages within one point of the PARCC-approved score. 
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4.2.3 Managing Scoring 


Pearson created a hand-scoring specifications document that detailed the hand-scoring schedule, 
customer requirements, rangefinding plans, quality management plans, item information, and staffing 
plans for each scoring administration. 


4.2.4 Monitoring Scoring 


Second Scoring 

During scoring, Pearson’s ePEN2 scoring system automatically and randomly distributed a minimum of 
10 percent of student responses for second scoring; scorers had no indication whether a response had 
been scored previously. Humans applied the second score for all mathematics items. Second scoring for 
ELA/L was performed either by human scorers or by the Intelligent Essay Assessor. If the first and second 
scores applied were non-adjacent, a third and occasionally a fourth score was assigned to resolve scorer 
disagreements. When a resolution score (i.e., third score) was nonadjacent to one and/or both of the 
first and second scores, the content specialist or Scoring Director would apply an adjudication score 


(fourth score). 


Table 4.3 Scoring Hierarchy Rules 


If a response was scored more than once, the following rules were applied to determine the final score: 


Score Type Rank Final Score Calculation 

Adjudication 1 If an adjudication score is assigned, this is the final score. 

Resolution 2 If no adjudication score is assigned, this is the final score. 

Backread 3 If no adjudication or resolution score is assigned, the latest 
backreading score is the final score. 

Human First Score 4 If no adjudication, resolution, or backreading score is assigned, 
this is the final score. 

Human Second Score 5 If no adjudication, resolution, backreading, or human first score 
is assigned, this is the final score. 

Intelligent Essay Assessor 6 If no human score is assigned, this is the final score. 

Score 


Backreading 

Backreading was one of the major responsibilities of Pearson Scoring Supervisors and a primary tool for 
proactively guarding against scorer drift, where scorers score responses in comparison to one another 
instead of in comparison to the training responses. Scoring supervisory staff used the ePEN2 
backreading tool to review scores assigned to individual student responses by any given scorer in order 
to confirm that the scores were correctly assigned and to give feedback and remediation to individual 
scorers. Pearson backread approximately 5 percent of the hand-scored responses. Backreading scores 
did not override the original score but were used to monitor scorer performance. 


Validity 
Validity responses are pre-scored responses strategically interspersed in the pool of live responses. 
These responses were not distinguishable from any other responses so that scorers were not aware they 
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were scoring validity responses rather than live responses. The use of validity responses provided an 
objective measure that helped ensure that scorers were applying the same standards throughout the 
project. In addition, validity was at times shared with scorers in a process known as “validity as review.” 
Validity as review provided scorers automated, immediate feedback: a chance to review responses they 
mis-scored, with reference to the correct score and a brief explanation of that score. One validity 
response was sent to scorers for every 25 “live” responses scored. 


PARCC validity agreement requirements for scorers are listed in Table 4.3. Scorers had to meet the 
required validity agreement percentages to continue working on the PARCC project. Scorers who did not 
maintain expected agreement statistics were given a series of interventions culminating in a targeted 
calibration set: a test of scorer knowledge. Scorers who did not pass targeted calibration were removed 
from scoring the item, and all the scores they assigned were deleted. 


Table 4.4 Scoring Validity Agreement Requirements 


Subject sal Perfect Agreement Within One Point 
Range 
Mathematics 0-1 90% 96% 
Mathematics 0-2 80% 96% 
Mathematics 0-3 70% 96% 
Mathematics 0-4 65% 95% 
Mathematics 0-6 65% 95% 
ELA/L Multi-trait 65% 96% 


*A zero or 1 score compared to a blank score will have a disagreement greater than 1 point. 


Calibration Sets 

Calibration sets are special sets created during scoring to help train scorers on particular areas of 
concern or focus. Scoring Directors used calibration sets to reinforce rangefinding standards, introduce 
scoring decisions, or address scoring issues and trends. Calibration was used either to correct a scoring 
issue or trend, or to continue scorer training by introducing a scoring decision. Calibration was 


administered regularly throughout scoring. 


Inter-rater Agreement 

Inter-rater agreement is the agreement between the first and second scores assigned to student 
responses and is the measure of how often scorers agree with each other. Pearson scoring staff used 
inter-rater agreement statistics as one factor in determining the needs for continuing training and 
intervention on both individual and group levels. PARCC inter-rater agreement expectations are shown 


in Table 4.4. 
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Table 4.5 Inter-rater Agreement Expectations and Results 


=e Within 
: Perfect Within One 
: Score Point Perfect Agreement : One 
Subject : Agreement Point ; 
Range Expectation . Point 
Result Expectation 
Result 
Mathematics 0-1 90% 97% 96%" 100% 
Mathematics 0-2 80% 94% 96% 100% 
Mathematics 0-3 70% 93% 96% 99% 
Mathematics 0-4 65% 92% 95% 99% 
Mathematics 0-5 65% 90% 95% 98% 
Mathematics 0-6 65% 91% 95% 97% 
ELA/L Multi-trait 65% 75% 96% 99% 


*A zero or 1 score compared to a blank score will have a disagreement greater than 1 point. 


Pearson’s ePEN2 scoring system included comprehensive inter-rater agreement reports that allowed 
supervisory personnel to monitor both individual and group performance. Based on reviews of these 
reports, scoring experts targeted individuals for increased backreading and feedback, and if necessary, 


retraining. 


The perfect agreement rate for all mathematics responses scored by two scorers was 93 percent and the 
within one point rate was 99 percent. For all ELA/L responses scored by two scorers, the perfect 
agreement rate was 75 percent and the within one point rate was 99.7 percent. 


The results by grade level for ELA/L are provided in Section 4.3.7: Inter-rater Agreement for Prose 


Constructed Response. 
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4.3 Automated Scoring for PARCC PCRs 


Automated scoring performed by Pearson’s Intelligent Essay Assessor (IEA) was the default option for 
scoring the PARCC assessment’s online prose constructed-response (PCR) tasks. Under the default 
option, it was assumed that operational scores for approximately 67 percent of the online PCR 
responses would be assigned by IEA for the spring administration. The operational scores for the 
remaining online responses were assigned by human scorers. Human scoring was applied to responses 
that were scored while IEA was being trained as well as to additional responses routed to human scoring 
when there was uncertainty about the automated scores. New Jersey adopted an option to have human 
scores assigned as the first score for 100 percent of responses for grades 3 through 8 and grade 10. 


For 10 percent of responses, a second “reliability” score was assigned. The purpose of the reliability 
score was to provide data for evaluating the consistency of scoring, which is done by evaluating scoring 
agreement. When IEA provided the first score of record, the second reliability score was a human score. 
For the New Jersey responses from grade levels with the human-scoring option, the second reliability 
score was assigned by IEA. 


4.3.1 Concepts Related to PARCC Automated Scoring 


The text below describes concepts related to PARCC automated scoring. 


Continuous Flow 

Continuous flow scoring results in an integrated connection between human scoring and automated 
scoring. It refers to a system of scoring where either an automated score, a human score, or both can be 
assigned based on a predetermined asynchronous operational flow. 


Calibration of IEA using Operational Data 

Continuous flow scoring facilitates the training of IEA using human scores assigned to operational online 
data collected early in the administration. Once IEA obtains sufficient data to train, it can be “turned on” 
and becomes the primary source of scoring (although human scoring continues for the 10 percent 
reliability sample and other responses that may be routed accordingly). 


Smart Routing 

Smart routing refers to the practice of using automated scoring results to detect responses that are 
likely to be challenging to score, and applying automated routing rules to obtain one or more additional 
human scores. Smart routing can be applied prompt by prompt to the extent needed to meet scoring 
quality criteria for automated scoring. 
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Quality Criteria for Evaluating Automated Scoring 

The PARCC state leads approved specific quality criteria for evaluating automated scoring at the time IEA 
was trained. The primary evaluation criteria for IEA was based on responses to validity papers with 
“known” scores assigned by experts. For each prompt scored, a set of validity papers is used to monitor 
the human-scoring process over time. Validity papers are seeded into human scoring throughout the 
administration. The expectation is that IEA can score validity papers at least as accurately as humans 
can. 


Additional measures of inter-rater agreement for evaluating automated scoring were proposed based 
on the research literature (Williamson, Xi, & Breyer 2012). These measures were previously utilized in 
Pearson’s automated scoring research and include Pearson correlation, kappa, quadratic-weighted 
kappa, exact agreement, and standardized mean difference. These measures are computed between 
pairs of human scores, as well as between IEA and humans, to evaluate how performance was the same 
or different. Criteria for evaluating the training of IEA given these measures include the following: 


e Pearson correlation between IEA-human should be within 0.1 of human-human. 

e Kappa between IEA-human should be within 0.1 of human-human. 

e Quadratic-weighted kappa between IEA-human should be within 0.1 of human-human. 
e Exact agreement between IEA-human should be within 5.25 percent of human-human. 
e Standardized mean difference between IEA-human should be less than 0.15. 


The specific criteria for evaluating IEA included both primary and secondary criteria and are noted 
below. 


e Primary Criteria—Based on responses to validity papers: With smart routing applied as needed, 
IEA agreement is as good as or better than human agreement for each trait score. 

e Contingent Primary Criteria—Based on the training responses if validity responses are not 
available: With smart routing applied as needed, [EA-human exact agreement is within 5.25 
percent of human-human exact agreement for each trait score. 

e Secondary Criteria—Based on the training responses: With smart routing applied as needed, IEA- 
human differences on statistical measures for each trait score are within the Williamson et al. 
tolerances for subgroups with at least 50 responses. 


Hierarchy of Assigned Scores for Reporting 
When multiple scores are assigned for a given response, the following hierarchy determines which score 
was reported operationally: 


e The IEA score is reported if it is the only score assigned. 

e If an IEA score and a human score are assigned, the human score is reported. 

e If two human scores are assigned, the first human score is reported. 

e If a backread score and human and/or IEA scores are assigned, the backread score is reported. 
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e If aresolution score is assigned and an adjudicated score is not assigned, the resolution score is 
reported (note that if nonadjacent scores are encountered, responses are automatically routed 
to resolution). 

e If an adjudicated score is assigned, it is reported (note that if a resolution score is nonadjacent 
to the other scores assigned, responses are automatically routed to adjudication). 


4.3.2 Sampling Responses Used for Training IEA 


For prompts trained using 2017 operational data, the early performance of human scoring was closely 
monitored to verify that an appropriate set of data would be available for training IEA. In particular, 
several characteristics of the human scoring data were monitored, including: 


e exact agreement between human scorers (the goal was for this to be at least 65 percent for 
each trait); 

e exact agreement between human scores conditioned on score point (the goal was for this to be 
at least 50 percent for each trait); 

e the number of responses at each score point (the goal was to have at least 40 responses at the 
highest score points in the training samples used by IEA); and 

e number of responses with two human scores assigned (note that IEA “ordered” additional 
scoring of responses during the sampling period as needed). 


Although the desired characteristics of the training data were easily achieved for some prompts, they 
were more challenging to achieve for others. For some prompts, a subset of scores were reset and 
clarifying directions were provided to scorers to improve human-human agreement. For other prompts, 
special sampling approaches were used to increase the numbers of responses that received top scores. 
In addition, a healthy percentage of responses were backread during the sampling period and these 
scores as well as double human scores were all part of the data used to train IEA. 


4.3.3 Primary Criteria for Evaluating IEA Performance 


The primary criteria for evaluating IEA performance is based on evaluating validity papers and is stated 
as follows: With smart routing applied as needed, IEA agreement is as good as or better than human 
agreement for each trait score. 


To operationalize the primary criteria for a given prompt, the following general steps are undertaken: 


Determine agreement of the human scores with the validity papers for each trait. 
Calculate agreement of the IEA scores with the validity papers for each trait. 
Compare the IEA validity agreement with the human agreement. 


a a i 


If the IEA validity agreement is greater than or equal to the human agreement for each trait, IEA 
can be deployed operationally. 
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In addition to looking at overall validity agreement, conditional agreement was also examined. In 
general, it was desirable for IEA to exceed 65 percent agreement at every score point as well as be close 
to or exceed the human validity agreement at each score point. 


4.3.4 Contingent Primary Criteria for Evaluating IEA Performance 


For many of the prompts trained in 2017, it was not possible to utilize human-scored validity responses 
in evaluating IEA performance. In these cases, IEA was evaluated based on IEA-human exact agreement 
for each trait score and compared to agreement based on responses that were double-scored by 
humans. A portion of the data was held out for evaluating IEA-human exact agreement according to the 
following steps: 


Determine exact agreement of the two human scores with each other for each trait. 
Calculate agreement of the IEA scores with the human scores for each trait. 
Compare the IEA-human agreement with the human-human agreement. 


grees oe 


If the IEA-human agreement is within 5.25 percent of the human-human agreement, IEA can be 
deployed operationally. 


In addition to the overall comparison, we targeted the following performance thresholds in the test data 
set: 1) at least 65 percent overall IEA-human agreement; and 2) 50 percent IEA-human agreement by 
score point (i.e., conditioned on the human score). These targets went beyond the contingent primary 
criteria approved by the state leads. 


4.3.5 Applying Smart Routing 


With smart routing, the quality of automated scoring can be increased by routing responses that are 
more likely to disagree with a human score to receive an additional human score. 


When human scorers read a paper, they typically apply integer scores based on a scoring rubric. When 
there is strong agreement between two independent human readers, they might both assign a score of 
3 such that the average score over both raters is also a 3 (i.e., (3+3)/2 = 3). IEA simulates this behavior, 
but because its scores come from an artificial intelligence algorithm, it generates continuous (i.e., 
decimalized) scores. In this case, the IEA score might be a 2.9 or 3.1. When human readers disagree on 
the score for a paper, say one reader gives the paper a score of 3 and another reader gives the paper a 
score of 4, the average of the two scores would be 3.5 (i.e., 3+4=7/2=3.5). For this paper, IEA would 
likely provide a score between 3 and 4, say 3.4 or 3.6. Because this continuous score needs to be 
rounded to an integer score for reporting, it might be reported as a 3 or a 4, depending on the rounding 
rules. Smart routing involves routing those responses with “in between” IEA scores to additional human 
scoring because the nature of the responses suggests there may be less confidence in the IEA score. 
Since these “in between” IEA scores are based on modeling human scores, it follows that human scores 
may be less certain as well, and thus such responses tend to be the ones that it makes sense to have 
double-scored and possibly to resolve if the IEA and human scores are non-adjacent. 
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Smart routing was utilized as needed to help IEA achieve targeted quality metrics (e.g., validity 
agreement or agreement with human scorers). Smart routing involved the application of the following 
four steps: 


1. The continuous IEA score for each of the two trait scores was rounded to the nearest score 
interval of 0.2, starting from zero. For example, IEA scores between 0 and 0.1 were rounded to 
an interval score of 0, scores between 0.1 and 0.3 were rounded to an interval score of 0.2, 
scores between 0.3 and 0.5 were rounded to an interval score of 0.4, and so on. 

2. Within each of these intervals, the percentage of exact agreement between IEA integer scores 
and the human scores was calculated for each trait. 

3. For each prompt, agreement rates were evaluated by rounding interval. Those intervals for 
which the agreement rates were below a designated threshold for either trait were identified. 

4. Once IEA scoring was implemented, responses within intervals for which IEA-human agreement 
was below the designated threshold were routed for additional human scoring. 


In training IEA, we first evaluated the scoring models without smart routing by applying either the 
primary validity criteria or the contingent criteria as described in Sections 4.2 and 4.3 of these 
procedures. For those prompts that did not meet these criteria, we applied increasing smart routing 
thresholds in an iterative fashion to filter scores and evaluate the remaining scores against the criteria. 
That is, in any one iteration a particular smart routing threshold was applied such that only scores falling 
in intervals for which exact agreement exceeded the threshold were included in evaluating the criteria. 
If the primary or contingent criteria were not met with this level of smart routing, we repeated the 
analysis applying an increased smart routing threshold. If the primary or contingent criteria were still not 
met, we repeated the analysis applying a still higher threshold. If the criteria were still not met after a 
maximum threshold was applied, we investigated different models and/or utilized additional human 
scoring data until an IEA scoring model was found that met the criteria. 


4.3.6 Evaluation of Secondary Criteria for Evaluating IEA Performance 


The secondary criteria for evaluating IEA performance involved comparing agreement indices for IEA- 
human scoring for various demographic subgroups. Because of the importance of protecting personally 
identifiable information (PII), student demographic data is stored and managed separately from the 
performance scoring data. For this reason, it was not possible to evaluate subgroup performance in real 
time as IEA was being trained. 


For those prompts trained on early operational data, attempts were made to prioritize the data being 
returned from the field to include data from states or districts where more diverse populations of 
students were anticipated. In addition, requests for additional human scores were made to increase the 
likelihood that there would be sufficient numbers of responses with two human scores for most of the 
demographic subgroups of interest. 
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Once IEA was trained and deployed, scoring sets used in training were matched to demographic 
information so that agreement between IEA and human scorers could be evaluated across subgroups. 
The analysis was conducted for the following ten comparison groups: 


Table 4.6 Comparison Groups 


Group Type Comparison Groups 

Sex Female 
Male 

Ethnicity American Indian/Alaska Native 
Asian 


Black/African American 
Hispanic/Latino 
Native Hawaiian or Other Pacific Islander 


White 
Special Instructional English Language Learners (ELL) 
Needs Students with Disabilities (SWD) 


IEA-human agreement indices were calculated for all cases with an IEA score and at least one human 
score. Human-human agreement was calculated for all cases with two human scores. 


To evaluate the training of IEA for subgroups, we applied the following criteria approved by the state 
leads for subgroups with at least 50 IEA-human scores and at least 50 human-human scores: 


e Pearson correlation between IEA-human should be within 0.1 of human-human. 

e Kappa between IEA-human should be within 0.1 of human-human. 

e Quadratic-weighted kappa between IEA-human should be within 0.1 of human-human. 

e Exact agreement between IEA-human should be within 5.25 percent of human-human. 

e Standardized mean difference between IEA-human should be less than +0.15 (this criterion was 
applied to subgroups with at least 50 IEA-human scores). 


Although it was not expected that these criteria would be met for all subgroups for all prompts, if results 
of the evaluation between IEA and human scoring for subgroups for any prompt indicated that IEA 
performance persistently failed on the criteria listed above, consideration would be given to resetting 
the responses scored by IEA and reverting to human scoring until such time that an alternate IEA model 
could be established with improved subgroup performance. 


In addition to the secondary criteria approved by the State Leads, we also compared the performance of 
IEA to the following targets on the various measures for subgroups with at least 50 responses: 


e Pearson correlation between IEA-human should be 0.70 or above. 

e Kappa between IEA-human should be 0.40 or above. 

e Quadratic-weighted kappa between IEA-human should be 0.70 or above. 
e Exact agreement between IEA-human should be 65 percent or above. 


March 27, 2018 Page 71 


PARCC 2017 Technical Report 


These targets were not intended to be directly applied in decisions about whether to deploy IEA 
operationally or not. Such targets may or may not be met by human scoring for any particular prompt 
and/or subgroup, and if they are not met by human scoring, they are unlikely to be met by IEA scoring. 
Nevertheless, comparisons to these targets provided additional information about IEA performance (and 
human scoring) in an absolute sense. 


4.3.7 Inter-rater Agreement for Prose Constructed Response 


This section presents the inter-rater agreement for operational results for the online prose constructed- 
response (PCR) tasks by trait and grade level in spring 2017. For the 2017 administration, the PCR task 
items were scored on two traits: (1) Reading Comprehension and Written Expression and (2) Knowledge 
of Language and Conventions. Narrative Writing tasks (NWT) continued to be scored on the same two 
traits as in 2015 and 2016: (1) Written Expression and (2) Knowledge of Language and Conventions. 


For 10 percent of responses, a second “reliability” score was assigned. The purpose of the reliability 
score was to provide data for evaluating the consistency of scoring, which is done by evaluating scoring 
agreement. Inter-rater agreement is the agreement between the first and second scores assigned to 
student responses and is the measure of how often scorers agree with each other. Pearson scoring staff 
used inter-rater agreement indices as one factor in determining the needs for continuing training and 
intervention on both individual and group levels. PARCC inter-rater agreement expectations are 
provided in Table 4.4 in Section 4.2.4. For ELA/L PCR traits, the expectation for agreement is an inter- 
rater agreement of 65 percent or higher between two scorers. When IEA provided the first score of 
record, the second reliability score was a human score. For those states choosing the human-scoring 
option, the second reliability score was assigned by IEA. For a subset of responses, the first and second 
score were both human scores. 


Table 4.5 presents the average agreement across the PCRs for each grade level by trait. The number of 
prompts included in the analyses is listed for each grade level. The agreement indices (exact agreement, 
kappa, quadratic-weighted kappa, and Pearson correlation) were calculated separately by PCR for each 
trait (Written Expression and Conventions). For each grade level, the agreement indices were averaged 
across the PCRs. The table presents the average count and the average for the agreement indices. 


The exact agreement for the PCR traits is above the 65 percent agreement rate criteria for all PCRs. The 
strength of agreement between raters is moderate to substantial agreement as defined by Landis and 
Koch (1977) for all PCRs. The quadratic-weighted kappa (QW Kappa) distinguishes between differences 
in ratings that are close to each other versus larger differences. The weighted kappa is substantial to 
almost perfect agreement for all grades. The Pearson correlations (r) ranged from .71 to .89. 


During operational scoring, the PCR agreement rates are monitored for quality and items not meeting 
the criteria are shared with the PARCC hand-scoring operational working group. After the operational 
administration, the performance of all the PCRs is provided to the content team as feedback for re-using 
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PCRs and in order to inform development of future PCRs. This provides evidence for continuous 


improvement of the testing program. 


Table 4.7 PARCC PCR Average Agreement Indices by Test 


Written Expression 


Conventions 


Number Qw QW 

Test of PCRs Count Exact Kappa Kappa r Exact Kappa Kappa r 

ELAO3 8 21,608 72.89 0.52 0.71 0.71 72.10 0.55 0.74 0.74 
ELA04 8 21,473 67.66 0.51 0.76 0.76 70.23 0.55 0.77 0.77 
ELAOS 8 20,393 72.61 0.55 0.77 0.77 71.23 0.57 0.80 0.80 
ELAO6 7 25,702 71.64 0.60 0.84 0.85 72.04 0.60 0.83 0.83 
ELAO7 7 24,904 71.93 0.61 0.87 0.87 70.74 0.60 0.85 0.85 
ELAO8 8 20,003 69.84 0.59 0.87 0.87 71.11 0.60 0.85 0.85 
ELAO9 6 12,155 73.22 0.64 0.89 0.89 74.22 0.65 0.87 0.87 
ELA10 8 11,004 73.44 0.64 0.88 0.88 73.33 0.63 0.86 0.86 
ELA11 8 5,123 73.58 0.63 0.88 0.88 73.51 0.63 0.87 0.87 
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Section 5: Test Taker Characteristics 


5.1 Overview of Test Taking Population 


Approximately three million students participated in the operational administration of the PARCC 
assessments during the 2016-2017 school year in Colorado, Bureau of Indian Education, District of 
Columbia, Illinois, Maryland, New Jersey, New Mexico, and Rhode Island. Not all participating states had 
students testing in all grades. Assessments were administered for English language arts/literacy (ELA/L) 
in grades 3 through 11; mathematics assessments were administered in grades 3 through 8, as well as 
for traditional high school mathematics (Algebra |, Geometry, and Algebra II) and integrated high school 
mathematics (Integrated Mathematics | — III). A small subset of students tested in ELA/L grades 9, 10, 
and 11, and Algebra |, Geometry, and Algebra II during fall of 2016. Test taker characteristics for this 
group are presented in an addendum.° The majority of students tested during the spring 2017 window 
when all grades and content areas were administered online and on paper. 


5.2 Composition of Operational Forms 


The fall 2016 and spring 2017 PARCC assessments were administered in either a computer-based test 
(CBT) or a paper-based test (PBT) format. ELA/L assessments focused on writing effectively when 
analyzing text. Mathematics assessments focused on applying skills and concepts, and feature multi-step 
problems that require abstract reasoning and modeling of real-world problems. Solving such problems 
requires the strategic use of tools, as well as precision and perseverance. In both content areas, 
students also demonstrated their acquired skills and knowledge by answering selected response items 
and fill-in-the-blank questions. Each assessment was comprised of multiple units; one of the 
mathematics units was split into calculator and non-calculator sections. 


5.3 Rules for Inclusion of Students in Analyses 


Criteria for inclusion of students were implemented prior to all operational analyses. These rules were 
established by Pearson psychometricians in consultation with PARCC to determine which, if any, student 
records should be removed from analyses. This data screening process resulted in higher quality, albeit 
slightly smaller, data sets. 


Student response data were included in analyses if: 


1) Valid form numbers were observed for each unit for online assessments or for the full form for 
paper assessments, 

2) Student records were not flagged as “void” (i.e., do not score), and 

3) The student attempted at least 25 percent of the items in each unit or form, 


> Addendum 5 presents a summary of the test taker characteristics for the fall 2016 administration. 
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Additionally, in cases where students had more than one valid record, the record with the higher raw 
score was chosen. Records for students with administration issues or anomalies were excluded from 
analyses. 


5.4 Test Takers by Grade, Mode, and Gender 


Table 5.1 presents, for each grade of ELA/L, the number and percentage of students who took the test in 
each mode (CBT or PBT). This information is provided for all participating states combined. Table 5.2 
presents the same type of information for all students who took the mathematics assessments, and 
Table 5.3 provides this information for students who took the mathematics assessments in Spanish. 


Markedly more students tested online than on paper across all grades for both content areas. For ELA/L, 
the percentages of online test takers, for all states combined, ranged from 86.4% to 98.9%, while the 
percentages of paper test takers ranged from 1.1% to 13.6%. For all mathematics test takers, the 
percentages of students testing online ranged from 86.4% to 99.3%, whereas the percentages of 
students testing on paper ranged from 0.7% to 13.6%. The percentages of students taking Spanish- 
language mathematics online forms ranged from 85.9% to 100% and the percentages of students taking 
paper Spanish-language mathematics paper forms ranged from 0.1% to 14.1%. Generally, the 
percentage of students who tested online increased steadily from the lower grades to the higher grades. 
For example, about 86% of the ELA/L grade 3 students tested online, while about 99% of the grade 11 
students tested online. Overall, fewer students tested at the higher grades for both content areas. 


Table 5.1 ELA/L Test Takers by Grade and Mode: All States Combined 


Gisae No. of Valid CBT PBT 

Cases N % N % 
420,496 363,304 86.4 57,192 13.6 
4 423,098 374,192 88.4 48,906 11.6 

5 414,461 399,511 96.4 14,950 3.6 

6 406,705 389,472 95.8 17,233 4.2 

7 406,201 390,667 96.2 15,534 3.8 

8 399,371 385,257 96.5 14,114 3.5 

9 188,201 185,251 98.4 2,950 1.6 
10 171,965 170,101 98.9 1,864 1.1 
11 107,593 106,447 98.9 1,146 1.1 

Grand Total 2,938,091 2,764,202 173,889 


Note: Includes students taking accommodated forms of ELA/L. 
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Table 5.2 Mathematics Test Takers by Grade and Mode: All States Combined 


Grade No. of Valid CBT PBT 
Cases N % N % 
3 425,744 367,726 86.4 58,018 13.6 
4 426,609 376,664 88.3 49,945 11.7 
5 416,556 400,631 96.2 15,925 3.8 
6 408,664 391,131 95.7 17,533 4.3 
7 393,942 379,753 96.4 14,189 3.6 
8 314,484 302,002 96.0 12,482 4.0 
Al 260,806 254,419 97.6 6,387 2.4 
GO 137,530 136,043 98.9 1,487 1.1 
A2 122,710 120,173 97.9 2,537 2.1 
M1 10,964 10,892 99.3 72 0.7 
M2 2,243 2,047 91.3 196 8.7 
M3 568 560 98.6 8 1.4 
Grand Total 2,920,820 2,742,041 178,779 


Note: Includes students taking mathematics in English, students taking 
Spanish-language forms for mathematics, and students taking accommodated 
forms. Al = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated 
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Mathematics |, M2 = Integrated Mathematics II, M3 = Integrated Mathematics III. 
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Table 5.3 Spanish-Language Mathematics Test Takers, by Grade and Mode: All States Combined 


Grade No. of Valid CBT PBT 
Cases N % N % 
3 6,399 5,498 85.9 901 14.1 
4 3,971 3,532 88.9 439 sie 
5 3,021 2,944 97.5 77 2.5 
6 2,736 2,694 98.5 42 1.5 
7 2,617 2,572 98.3 45 1,7 
8 2,573 2,532 98.4 41 1.6 
Al 4,032 3,646 90.4 386 9.6 
GO 1,755 1,755 100 n/a n/a 
A2 1,085 1,084 99.9 1 0.1 
MI 88 88 100 n/a n/a 
M2 1 1 100 n/a n/a 
M3 1 1 100 n/a n/a 
Grand Total 28,279 26,347 1,932 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra Il, M1 = Integrated 
Mathematics |, M2 = Integrated Mathematics II, M3 = Integrated Mathematics 
Ill. n/a = not applicable. 


Tables A.5.1, A.5.2, and A.5.3 in Appendix 5 show the number and percentage of students with valid test 
scores in each content area (including Spanish Language mathematics), grade, and mode of assessment 
for all states combined and for each state separately. Tables A.5.4, A5.5, and A5.6 present the 
distribution by content area, grade, mode, and gender, for all states combined. 


5.5 Demographics 


Also presented in Appendix 5 is student demographic information for the following characteristics: 
economically disadvantaged, students with disabilities, English learners (EL), gender, and race/ethnicity 
(American Indian/Alaska Native; Asian; Black/African American; Hispanic/Latino; White/Caucasian; 
Native Hawaiian or Other Pacific Islander; two or more races reported; race not reported). Student 
demographic information was provided by the states and districts and captured in PearsonAccess"™™ by 
means of a student data upload. The demographic data was verified by the states and districts prior to 
score reporting. 


Tables A.5.7 through A.5.15 provide demographic information for students with valid ELA/L scores, and 
Tables A.5.16 through A.5.27 present demographics for students with valid mathematics scores. All 
tables of demographic information are organized by grade; the results are first aggregated across all 
PARCC states and then presented for each state. Percentages are not reported for any states in which 
fewer than 20 students tested in a grade/content area. 
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Section 6: Classical Item Analysis 


6.1 Overview 


This section describes the results of the classical item analysis conducted for data obtained from the 
operational test items. For the spring 2017 administration, mathematics online and paper and ELA/L 
paper tests were pre-equated; ELA/L online tests were post-equated. For pre-equated tests, the item 
statistics provided in this section are from prior operational administrations and reflect the statistics 
that were used at test construction. For the post-equated tests, the statistics provided in this section are 
from the spring 2017 administration. Item analysis serves two purposes: to inform item exclusion 
decisions for IRT analysis and to provide item statistics for the item bank. 


PARCC item analysis included data from the following types of items: key-based selected-response 
items, rule-based machine-scored items, and hand-scored constructed-response items. For each item, 
the analysis produced item difficulty, item discrimination, and item response frequencies. 


6.2 Data Screening Criteria 


Item analyses were conducted by test form based on administration mode. In preparation for item 
analysis, student response files were processed to verify that the data were free of errors. Pearson 
Customer Data Quality (CDQ) staff ran predefined checks on all data files and verified that all fields and 
data needed to perform the statistical analyses were present and within expected ranges. 


Before beginning item analysis, Pearson performed the following data screening operations: 


All records with an invalid form number were excluded. 

All records that were flagged as “void” were excluded. 

All records where the student attempted fewer than 25 percent of items were excluded. 

For students with more than one valid record, the record with the higher raw score was chosen. 


Le a a 


Records for students with administration issues or anomalies were excluded. 
6.3 Description of Classical Item Analysis Statistics 


A set of classical item statistics were computed for each operational item by form and by administration 
mode. Each statistic was designed to evaluate the performance of each item. 


The following statistics and associated flagging rules were used to identify items that were not 
performing as expected: 


Classical item difficulty indices (p value and average item score) 

When constructing PARCC tests, a wide range of item difficulties is desired (i.e., from easy to hard items) 
so that students of all ability levels can be assessed with precision. At the operational stage, item 
difficulty statistics are used by test developers to build forms that meet desired test difficulty targets. 
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For dichotomously scored items, item difficulty is indicated by its p value, which is the proportion of test 
takers who answered that item correctly. The range for p values is from .00 to 1.00. Items with high p 
values are easy items and those with low p values are difficult items. Dichotomously scored items were 
flagged for review if the p value was above .95 (i.e., too easy) or below .20 (i.e., too difficult). 


For polytomously scored items, difficulty is indicated by the average item score (AIS). The AIS can range 
from .00 to the maximum total possible points for an item. To facilitate interpretation, the AIS values for 
polytomously scored items are often expressed as percentages of the maximum possible score, which 
are equivalent to the p values of dichotomously items. The desired p value range for polytomously 
scored items is .30 to .80; items with values outside this range were flagged for review. 


The percentage of students choosing each response option 

Selected-response items on PARCC assessments refer primarily to single-select multiple choice scored 
items. These items require that the test taker select a response from a number of answer options. These 
statistics for single-select multiple choice items indicate the percentage of students who select each of 
the answer options and the percentage that omit the item. The percentages are also computed for the 
high-performing subgroup of students who scored at the top 20 percent on the assessment. Items were 
flagged for review if more high-performing test takers chose the incorrect option than the correct 
response. Such a result could indicate that the item has multiple correct answers or is miskeyed. 


Item-total correlation 

This statistic describes the relationship between test takers’ performance on a specific item and their 
performance on the total test. The item-total correlation is usually referred to as the item discrimination 
index. For PARCC operational item analysis, the total score on the assessment was used as the total test 
score. The polyserial correlation was calculated for both selected-response items and constructed- 
response items as an estimate of the correlation between an observed continuous variable and an 
unobserved continuous variable hypothesized to underlie the variable with ordered categories (Olsson, 
Drasgow, and Dorans, 1982). Item-total correlations can range from -1.00 to 1.00. Desired values are 
positive and larger than .15. Negative item-total correlations indicate that low-ability test takers 
perform better on an item than high-ability test takers, an indication that the item may be potentially 
flawed. Item-total correlations below .15 were flagged for review. Items with extremely low or negative 
values were considered for exclusion from IRT calibrations or linking (refer to Section 10 for details on 
item inclusion and exclusion criteria for IRT analyses). 


Distractor-total correlation 

For selected-response items, this estimate describes the relationship between selecting an incorrect 
response (i.e., a distractor) for a specific item and performance on the total test. The item-total 
correlation is calculated (refer to #3) for the distractors. Items with distractor-total correlations above 
.0O were flagged for review as these items may have multiple correct answers, be miskeyed, or have 
other content issues. 
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Percentage of students omitting or not reaching each item 

For both selected-response and constructed-response items, this statistic is useful for identifying 
problems with test features such as testing time and item/test layout. Typically, if students have an 
adequate amount of testing time, approximately 95 percent of students should attempt to answer each 
question on the test. A distinction is made between “omit” and “not reached” for items without 
responses. 


e Anitem is considered “omit” if the student responded to subsequent items. 
e Anitem is considered “not reached” if the student did not respond to any subsequent 
items. 


Patterns of high omit or not-reached rates for items located near the end of a test section may indicate 
that test takers did not have adequate time. Items with high omit rates were flagged. Omit rates for 
constructed-response items tend to be higher than for selected-response items. Therefore, the omit 
rate for flagging individual items was 5 percent for selected-response items and 15 percent for 
constructed-response items. If a test taker omitted an item, then the test taker received a score of 0 for 
that item and was included in the N-count for that item. However, if an item was near the end of the 
test and classified as not reached, the test taker did not receive a score and was not included in the N- 
count for that item. 


Distribution of item scores 

For constructed-response items, examination of the distribution of scores is helpful to identify how well 
the item is functioning. If no students’ responses are assigned the highest possible score point, this may 
indicate that the item is not functioning as expected (e.g., the item could be confusing, poorly worded, 
or just unexpectedly difficult), the scoring rubric is flawed, and/or test takers did not have an 
opportunity to learn the content. In addition, if all or most test takers score at the extreme ends of the 
distribution (e.g., O and 2 for a 3-category item), this may indicate that there are problems with the item 
or the rubric so that test takers can receive either full credit or no credit at all, but not partial credit. 


The raw score frequency distributions for constructed-response items were computed to identify items 
with few or no observations at any score points. Items with no observations or a low percentage (i.e., 
less than 3 percent) of test takers obtaining any score point were flagged. In addition, constructed- 
response items were flagged if they had U-shaped distributions, with high frequencies for extreme 
scores and very low frequencies for middle score categories. Items with such response patterns may 
pose problems during the IRT calibrations and therefore may need to be excluded (refer to Section 10 
for more information). 


6.4 Summary of Classical Item Analysis Flagging Criteria 
In summary, items are flagged for review if the item analysis yielded any of the following results: 


1. p-value above .95 for dichotomous items and above .80 for polytomous items 
2. p value below .25 for dichotomous items, and below .30 for polytomous items 
3. item-total correlation below .15 
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4. any distractor-total correlation above .00 

5. greater number of high-performing students (top 20 percent) choosing a distractor rather than 
the keyed response 

6. high percentage of omits: above 5 percent for selected-response items and above 15 percent for 
constructed-response items 

7. high percentage that did not reach the item: above 5 percent for selected-response items and 
above 15 percent for constructed-response items 

8. constructed-response items with a score value obtained by less than 3 percent of responses 


Pearson’s psychometric staff carefully reviewed the flagged items and brought items to the PARCC 
Priority Alert Task Force to decide if the items were problematic and should be excluded from scoring. 


6.5 Classical Item Analysis Results 


This section presents tables summarizing the analyses for items on the spring 2017 operational forms. 
Spring 2017 PARCC assessments (except ELA/L online) were pre-equated, meaning that the scoring was 
based on item parameters estimated using data from earlier administrations. For the pre-equated 
grades/subjects, item analysis results in this section are the item statistics from prior administrations 
that were used to make decisions during test construction and for scoring. For ELA/L online, the item 
analysis is from the spring 2017 operational administration. 


e Tables 6.1 and 6.2 present p value information by grade and mode for the ELA/L and 
mathematics operational items. 

e Tables 6.3 and 6.4 present item-total correlations by grade and mode for the ELA/L and 
mathematics operational items. 


An operational item could appear on multiple test forms. The tables list only unique items in each test 
mode, and the reported item statistics are based on student responses across multiple occurrences of 
an item. 


Spoiled or “do not score” items were excluded from the total test score in item analysis. These items 
were removed from scoring because of item performance, technical scoring issues, content concerns, or 
multiple/no correct answers. Additionally, some items were dropped during item calibrations due to: 


e alow weighted polyserial, 
e alow p value (e.g., extremely difficult item), or 
e extremely poor IRT model fit or item not able to calibrate. 


Table 10.2 in section 10 presents the count and percentage of items excluded from IRT calibration along 
with the reasons the items were excluded for ELA/L. The tables in this section and in the Appendix 
include only those items that were used for operational scoring. 


The fall 2016 forms were based on the spring 2016 operational forms; therefore, the item analyses for 
these forms were reported in the 2015-2016 Technical Report. 
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Table 6.1 Summary of p Values for ELA/L Operational Items by Grade and Mode 


es Mode cae Mean SD Min Max Median 
ieamme p Value p Value p Value p Value p Value 
3 CBT 82 0.45 0.18 0.14 0.84 0.46 
3 PBT 32 0.39 0.14 0.13 0.75 0.37 
4 CBT 86 0.46 0.16 0.14 0.78 0.46 
4 PBT 34 0.43 0.14 0.13 0.72 0.41 
5 CBT 85 0.42 0.15 0.09 0.80 0.38 
S PBT 34 0.39 0.12 0.17 0.78 0.37 
6 CBT 106 0.43 0.13 0.15 0.71 0.42 
6 PBT 40 0.44 0.12 0.22 0.69 0.41 
7 CBT 98 0.44 0.14 0.17 0.79 0.42 
7 PBT 40 0.44 0.13 0.21 0.71 0.40 
8 CBT 104 0.46 0.14 0.20 0.78 0.44 
8 PBT 40 0.48 0.14 0.31 0.84 0.46 
9 CBT 96 0.43 0.12 0.18 0.74 0.42 
9 PBT 40 0.43 0.14 0.19 0.76 0.43 
10 CBT 104 0.40 0.12 0.13 0.71 0.39 
10 PBT 40 0.43 0.13 6.21 0.70 0.42 
11 CBT 106 0.35 0.11 0.12 0.61 0.34 
11 PBT 40 0.35 0.10 0.19 0.62 0.32 


Note: CBT = computer-based testing (online); PBT = paper-based testing (paper). 
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Table 6.2 Summary of p Values for Mathematics Operational Items by Grade and Mode 


Grade Mode eee Mean SD Min Max Median 
iccae p Value p Value p Value p Value p Value 
3 CBT 113 0.50 0.22 0.06 0.94 0.49 
3 PBT 43 0.54 0.20 0.14 0.89 0.52 
4 CBT 99 0.46 0.19 0.07 0.89 0.47 
4 PBT 40 0.48 0.22 0.11 0.86 0.51 
5 CBT 103 0.44 0.20 0.08 0.88 0.44 
5 PBT 40 0.50 0.23 0.08 0.88 0.54 
6 CBT 95 0.42 0.22 0.09 0.95 0.38 
6 PBT 38 0.44 0.21 0.13 0.93 0.43 
7 CBT 97 0.30 0.18 0.05 0.75 0.26 
7 PBT 38 0.34 0.21 0.08 0.83 0.30 
8 CBT 91 0.30 0.20 0.06 0.86 0.28 
8 PBT 36 0.36 0.21 0.04 0.74 0.34 
Al CBT 104 0.22 0.13 0.02 0.64 0.18 
Al PBT 42 0.34 0.19 0.01 0.71 0.34 
GO CBT 102 0.23 0.13 0.04 0.57 0.21 
GO PBT 41 0.27 0.18 0.05 0.73 0.22 
A2 CBT 110 0.28 0.17 0.03 0.79 0.25 
A2 PBT 43 0.32 0.19 0.04 0.74 0.30 
M1 CBT 42 0.31 0.18 0.03 0.74 0.27 
M1 PBT 42 0.33 0.21 0.01 0.84 0.37 
M2 CBT 41 0.25 0.17 0.01 0.62 0.22 
M2 PBT 42 0.29 0.17 0.01 0.72 0.28 
M3 CBT 40 0.28 0.17 0.04 0.82 0.26 
M3 PBT 39 0.31 0.19 0.03 0.83 0.29 


Note: Al = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics II], M3 = Integrated Mathematics Ill. CBT = computer-based testing (online); PBT = paper- 
based testing (paper). 


March 27, 2018 Page 83 


PARCC 2017 Technical Report 


Table 6.3 Summary of Item-Total Polyserial Correlations for ELA/L Operational Items by Grade and Mode 


Grade Mode cae Meat =e : wn : mes , Mee 
hers Polyserial  Polyserial Polyserial Polyserial Polyserial 
3 CBT 82 0.53 0.12 0.18 0.75 0.54 
3 PBT 32 0.53 0.13 0.25 0.73 0.54 
4 CBT 86 0.50 0.15 0.19 0.81 0.48 
4 PBT 34 0.50 0.15 0.19 0.79 0.47 
5 CBT 85 0.49 0.16 0.19 0.79 0.48 
5 PBT 34 0.47 0.17 0.20 0.77 0.44 
6 CBT 106 0.49 0.14 0.20 0.81 0.47 
6 PBT 40 0.50 0.15 0.21 0.80 0.51 
7 CBT 98 0.48 0.16 0.22 0.84 0.45 
7 PBT 40 0.47 0.13 0.28 0.77 0.43 
8 CBT 104 0.48 0.17 0.13 0.85 0.46 
8 PBT 40 0.47 0.16 0.17 0.81 0.47 
9 CBT 96 0.48 0.16 0.19 0.85 0.47 
9 PBT 40 0.49 0.14 0.27 0.79 0.48 
10 CBT 104 0.48 0.17 0.21 0.86 0.44 
10 PBT 40 0.53 0.14 0.28 0.83 0.53 
11 CBT 106 0.49 0.16 0.21 0.83 0.47 
11 PBT 40 0.45 0.18 0.21 0.82 0.41 


Note: CBT = computer-based testing (online); PBT = paper-based testing (paper). 


March 27, 2018 Page 84 


PARCC 2017 Technical Report 


Table 6.4 Summary of Item-Total Correlations for Mathematics Operational Items by Grade and Mode 


Grade Mode cae Mean =e . wp : eae Meolan 
iene Polyserial Polyserial Polyserial Polyserial Polyserial 
3 CBT 113 0.54 0.14 0.18 0.80 0.55 
3 PBT 43 0.53 0.13 0.24 0.80 0.54 
4 CBT 99 0.57 0.11 0.29 0.79 0.58 
4 PBT 40 0.55 0.12 0.30 0.79 0.57 
5 CBT 103 0.52 0.13 0.22 0.81 0.53 
5 PBT 40 0.53 0.12 0.34 0.81 0.53 
6 CBT 95 0.54 0.16 0.20 0.88 0.57 
6 PBT 38 0.53 0.16 0.17 0.80 0.53 
7 CBT 97 0.53 0.16 0.20 0.91 0.55 
7 PBT 38 0.50 0.15 0.25 0.96 0.47 
8 CBT 91 0.49 0.13 0.22 0.89 0.50 
8 PBT 36 0.49 0.14 0.22 0.84 0.50 
Al CBT 104 0.51 0.17 0.21 0.90 0.51 
Al PBT 42 0.46 0.14 0.16 0.72 0.46 
GO CBT 110 0.53 0.16 0.18 0.98 0.53 
GO PBT 43 0.51 0.15 0.23 0.78 0.53 
A2 CBT 102 0.47 0.13 0.16 0.83 0.46 
A2 PBT 41 0.45 0.16 0.10 0.75 0.47 
M1 CBT 42 0.53 0.16 0.19 0.83 0.53 
M1 PBT 42 0.47 0.16 0.15 0.8 0.46 
M2 CBT 41 0.4 0.13 0.16 0.67 0.38 
M2 PBT 42 0.43 0.17 0.16 0.76 0.38 
M3 CBT 40 0.46 0.16 0.20 0.75 0.47 
M3 PBT 39 0.44 0.16 0.12 0.74 0.44 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics Il, M3 = Integrated Mathematics Ill. CBT = computer-based testing (online); PBT = paper- 
based testing (paper) 
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Section 7: Differential Item Functioning 


7.1 Overview 


Differential item functioning (DIF) analyses were conducted using the data obtained from the 
operational items. If an item performs differentially across identifiable subgroups (e.g., gender or 
ethnicity) when students are matched on ability, the item may be measuring something other than the 
intended construct (i.e., possible evidence of DIF). It is important, however, to recognize that item 
performance differences flagged for DIF might be related to actual differences in relevant knowledge or 
skills (item impact) or statistical Type | error. As a result, DIF statistics are used to identify potential item 
bias. Subsequent reviews by content experts and bias/sensitivity committees are required to determine 
the source and meaning of performance differences. 


In this section, DIF statistics are presented for the spring 2017 ELA/L online post-equated tests. In 
addition, the DIF statistics used at test construction to make decisions about items are provided for pre- 
equated grades/subjects: mathematics online and paper and ELA/L paper. 


7.2 DIF Procedures 


Dichotomous Items 

The Mantel-Haenszel (MH) DIF statistic was calculated for selected-response items and for 
dichotomously scored constructed-response items. In this method, test takers are classified to relevant 
subgroups of interest (e.g., gender or ethnicity). Using the PARCC raw score total as the criteria, test 
takers in a certain total score category in the focal group (e.g., females) are compared with examinees in 
the same total score category in the reference group (e.g., males). For each item, test takers in the focal 
group are also compared to test takers in the reference group who performed equally well on the test as 
a whole. The common odds ratio is estimated across all categories of matched test taker ability using 
the following formula (Dorans & Holland, 1993), and the resulting estimate is interpreted as the relative 
likelihood of success on a particular item for members of two groups when matched on ability. 


Quan = 7 2 (7-1) 


in which: 


S =the number of score categories, 

R,s= the number of test takers in the reference group who answer the item correctly, 

W;; = the number of test takers in the focal group who answer the item incorrectly, 

Rys = the number of test takers in the focal group who answer the item correctly, 

W,;= the number of test takers in the reference group who answer the item incorrectly, and 
Nis = the total number of test takers. 
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To facilitate the interpretation of MH results, the common odds ratio is frequently transformed to the 
delta scale using the following formula (Holland & Thayer, 1988): 


MH D-DIF = -2.35 In(¢@,,, ) (7-2) 


Positive values indicate DIF in favor of the focal group (i.e., positive DIF items are differentially easier for 
the focal group), whereas negative values indicate DIF in favor of the reference group (i.e., negative DIF 
items are differentially easier for the reference group). 


Polytomous Items 

For polytomously scored constructed-response items, the MH D-DIF statistic is not calculated; instead 
the standardization DIF (Dorans & Schmitt, 1991; Zwick, Thayer & Mazzeo, 1997; Dorans, 2013), in 
conjunction with the Mantel chi-square statistic (Mantel, 1963; Mantel & Haenszel, 1959), is used to 
identify items with DIF. 


The standardization DIF compares the item means of the two groups after adjusting for differences in 
the distribution of test takers across the values of the matching variable (i.e., total test score) and is 
calculated using the following formula: 


Wea NpstEp(Y|X = S) _ Dei Nps*Er(Y|X = S) _ yS_, Ds 


STD — EISDIF = 55 Nps De Nys sai N es’ 


(7-3) 


in which: 


X= the total score 

Y= the item score 

S =the number of score categories on X, 

N,s= the number of test takers in the reference group in score category s, 
Nys= the number of test takers in the focal group in score category s, 

E,= the expected item score for reference group, and 

E;= the expected item score for focal group. 


A positive STD-EISDIF value means that, conditional on the total test score, the focal group has a higher 
mean item score than the reference group. In contrast, a negative STD-EISDIF value means that, 
conditional on the total test score, the focal group has a lower mean item score than the reference 
group. 


Classification 

Based on the DIF statistics and significance tests, items are classified into three categories and assigned 
values of A, B, or C (Zieky, 1993). Category A items contain negligible DIF, Category B items exhibit slight 
to moderate DIF, and Category C items possess moderate to large DIF values. Positive values indicate 
that, conditional on the total score, the focal group has a higher mean item score than the reference 
group. In contrast, negative DIF values indicate that, conditional on the total test score, the focal group 
has a lower mean item score than the reference group. The flagging criteria for dichotomously scored 
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items are presented in Table 7.1; the flagging criteria for polytomously scored constructed-response 
items are provided in Table 7.2. 


Table 7.1 DIF Categories for Dichotomous Selected-Response and Constructed-Response Items 


DIF Category Criteria 


Absolute value of the MH D-DIF is not significantly different from zero, or is 


A (negligible) less than one 


1. Absolute value of the MH D-DIF is significantly different from zero but not 
from one, and is at least one; or 

B (slight to moderate) 2. Absolute value of the MH D-DIF is significantly different from one, but is 
less than 1.5. 
Positive values are classified as “B+” and negative values as “B-”. 


Absolute value of the MH D-DIF is significantly different from one, and is at 


C (moderate to large a ee ie ; uc 
( ee) least 1.5. Positive values are classified as “C+” and negative values as “C-”. 


Table 7.2 DIF Categories for Polytomous Constructed-Response Items 


DIF Category Criteria 


A (negligible) Mantel Chi-square p value > 0.05 or |STD-EISDIF/SD| < 0.17 


B (slight to moderate) Mantel Chi-square p value < 0.05 and | STD-EISDIF/SD| > 0.17 


C (moderate to large) Mantel Chi-square p value < 0.05 and | STD-EISDIF/SD| > 0.25 


Note: STD-E/SDIF = standardized DIF; SD = total group standard deviation of item score. 


7.3 Operational Analysis DIF Comparison Groups 


Traditional Comparisons 

DIF analyses were conducted on each test form for designated comparison groups defined on the basis 
of demographic variables including: gender, race/ethnicity, economic disadvantage, and special 
instructional needs such as students with disabilities (SWD) or English learners (EL). Student 
demographic information was provided by the states and district and captured in PearsonAccess by 
means of a student data upload. The demographic data was verified by the states and district prior to 
score reporting. These comparison groups are specified in Table 7.3. 
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Table 7.3 Traditional DIF Comparison Groups 


Grouping Variable Focal Group Reference Group 
Gender Female Male 
= American Indian/Alaska Native ; 

Ethnicity ; White 
(Amerindian) 
Asian White 
Black or African American White 
Hispanic/Latino White 
Native Hawaiian or Pacific Islander White 
Multiple Race Selected White 

Economic Status” Economically Disadvantaged (EcnDis) | Not Economically Disadvantaged 

(NoEcnDis) 


Special Instructional 


English Learner (ELY) Non English Learner (ELN) 
Needs 


Students without Disabilities 


Students with Disabilities (SWDY) 
(SWDN) 


Note: * Economic status was based on participation in National School Lunch Program (receipt of free or 
reduced-price lunch). 


Comparison across Languages 

DIF analyses were also conducted for Spanish-language items vs. English-language items in mathematics 
for items that previously were not evaluated for Spanish-language DIF. The purpose of the Spanish vs. 
English DIF analysis was to evaluate how similarly the items functioned between the two languages 
because the data from the Spanish-language forms were not separately calibrated using IRT. The item 
parameter estimates based on the English-speaking test takers were used to generate conversion tables 
for the Spanish-language forms. Spanish-language mathematics items flagged for C-DIF were reviewed 
by content specialists and the PARCC Priority Alert Task Force to decide if the items were problematic 
and should be excluded from scoring. An item could be dropped from a Spanish-language form but 
remain in the English-language form if no other issues were detected; in those cases, separate 
conversion tables were generated for the two versions of the form that had different numbers of items. 


The Spanish-language forms did not have a non-accommodated English-language form counterpart with 
the same set of items (refer to Section 2 for more information on the development of Spanish-language 
forms). Most of the Spanish-language items were previously evaluated for Spanish-language DIF. For 
items that had not been evaluated for Spanish-language DIF, the analyses were conducted for items that 
had an English-language item on a non-accommodated test form. 


Sample Size Requirement 
DIF analyses were conducted when the following sample size requirements were met: 


e the smaller group, reference or focal, had at least 100 students, and 
e the combined group, reference and focal, had at least 400 students. 
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Spanish-language vs. English-language DIF analyses were not conducted for Integrated Mathematics |, II, 
and III because of insufficient sample sizes. 


7.4 Operational Differential Item Functioning Results 


Appendix 7 presents tables summarizing the DIF results for the spring 2017 ELA/L post-equated online 
operational items as well as the pre-equated item DIF results that were used to inform decisions at test 
construction. There is one table prepared for each content and grade level (e.g., ELA/L Grade 3). The fall 
2016 forms were based on spring 2016 operational forms. The DIF analyses for these forms are reported 
in the 2015-2016 Technical Report. 


Spoiled or “do not score” items were excluded from the total test score for each form in DIF analysis. 
These items were removed from scoring because of item performance, technical scoring issues, content 
concerns, multiple correct answers, or no correct answers. However, the tables in this section may 
include items for certain grade levels that were excluded from scoring based on later analyses (refer to 
Section 10.5 Items Excluded from Score Reporting for more information). 


In the DIF results tables, the column “DIF Comparisons” identifies the focal and reference groups for the 
analysis performed; the column “Mode” identifies the test delivery mode. “Total N of Unique Items” 
reports the number of unique items included in the analysis, whereas “Total N of Item Occurrences” 
reports the number of times items were used on test forms. An item could be used in multiple test 
forms; therefore, items were counted according to the occurrences. For example, if the same item 
appeared in five test forms, it was counted as five occurrences; if this item was classified as B+ on one 
form and C+ on another form, both occurrences were reported in the corresponding columns. For the 
Spanish-language DIF, “Total N of Item Occurrences” reports the number of items previously not 
analyzed for Spanish-language DIF and a non-accommodated English-language item was available in 
spring 2017. “Total N of Item Occurrences Included in DIF Analysis” reports the number of occurrences 
with sufficient sample sizes to be included in DIF analyses. In addition, “O” indicates that the DIF analysis 
did not classify any items in the particular DIF category, while “n/a” indicates that the DIF analysis was 
not performed due to insufficient sample sizes. 
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Table 7.4 Differential Item Functioning for ELA/L Grade 3 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode . Item Occurrences N of N of N of N of N of 
Unique . Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences. Occurrences | Occurrences’. Occurrences’. Occurrences’. 
Items . in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 

Male vs Female CBT 90 105 105 1 1 102 97 2 2 

PBT 29 35 35 35 100 
White vs Amerindian CBT 90 105 105 1 1 104 99 

PBT 29 35 35 2 6 33 94 
White vs Asian CBT 90 105 105 104 99 1 1 

PBT 29 35 35 34 97 1 3 
White vs Black CBT 90 105 105 iq 1 104 99 

PBT 29 35 35 1 3 34 97 
White vs Hispanic CBT 90 105 105 105 100 

PBT 29 35 35 3 9 32 91 
White vs Pacific CBT 
Islander 90 105 105 2 2 103 98 

PBT 29 35 35 33 94 
White vs Multiracial CBT 90 105 105 105 100 

PBT 29 35 35 35 100 
NoEcnDis vs EcnDis CBT 90 105 105 105 100 

PBT 29 35 35 35 100 
ELN vs ELY CBT 90 105 105 2 2 103 98 

PBT 29 35 35 35 100 
SWDN vs SWDY CBT 90 105 105 105 100 

PBT 29 35 35 1 3 34 97 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Total N Total N of Laas ou C- DIF B- DIF A DIF B+ DIF C+ DIF 
DIF Comparisons Mode Unique Item Deciirenees ice % of Total % of Total % of Total % of Total % of Total 
Items ae hi a 7 Occurrences oy pinta Occurrences coe Occurrences menurnenite Occurrences REMIT BR Occurrences eee 
ysis in DIF in DIF in DIF in DIF in DIF 
Male vs Female CBT 113 124 124 3 2 120 97 1 1 
PBT 43 63 63 1 2 62 98 
White vs Amerindian CBT 113 124 124 3 2 1 1 118 95 2 2 
PBT 43 63 63 1 2 3 5 57 90 2 3 
White vs Asian CBT 113 124 124 1 1 107 86 13 10 3 2 
PBT 43 63 63 60 95 2 3 1 2 
White vs Black CBT 113 124 124 5 4 5 4 105 85 9 7 
PBT 43 63 63 3 5 58 92 2 3 
White vs Hispanic CBT 113 124 124 4 3 119 96 1 1 
PBT 43 63 63 3 5 58 92 2 3 
White vs Pacific 
Islander CBT 113 124 124 123 99 1 1 
PBT 43 63 63 59 94 4 6 
White vs Multiracial CBT 113 124 124 al 1 121 98 2 2 
PBT 43 63 63 62 98 1 2 
NoEcnDis vs EcnDis CBT 113 124 124 123 99 1 1 
PBT 43 63 63 61 97 1 2 1 2 
ELN vs ELY CBT 113 124 124 2 2 122 98 
PBT 43 63 63 1 2 62 98 
SWDN vs SWDY CBT 113 124 124 2 2 3 2 117 94 2 2 
PBT 43 63 63 1 2 61 97 1 2 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. Small sample sizes may result in fewer items in the column “Total N of Item Occurrences Included in DIF Analysis. 
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Section 8: Reliability 


8.1 Overview 


Reliability focuses on the extent to which differences in test scores reflect true differences in the 
knowledge, ability, or skill being tested rather than fluctuations due to chance. Thus, reliability measures 
the consistency of the scores across conditions that can be assumed to differ at random, especially 
which form of the test the test taker is administered and which persons are assigned to score responses 
to constructed-response questions. In statistical terms, the variance in the distributions of test scores, 
essentially the differences among individuals, is partly due to real differences in the knowledge, skill, or 
ability being tested (true variance) and partly due to random errors in the measurement process (error 
variance). Reliability is an estimate of the proportion of the total variance that is true variance. 


There are several different ways of estimating reliability. The type of reliability estimate reported here is 
an internal-consistency measure, which is derived from analysis of the consistency of the performance 
of individuals across items within a test. It is used because it serves as a good estimate of alternate 
forms reliability, but it does not take into account form-to-form variation due to lack of test form 
parallelism, nor is it responsive to day-to-day variation due to, for example, the examinee’s state of 
health or the testing environment. 


Reliability coefficients range from 0 to 1. The higher the reliability coefficient for a set of scores, the 
more likely individuals would be to obtain very similar scores upon repeated testing occasions, if the 
students do not change in their level of the knowledge or skills measured by the test. The reliability 
estimates in the tables to follow attempt to answer the question, “How consistent would the scores of 
these test takers be over replications of the entire testing process?” 


Reliability of classification estimates the proportion of students who are accurately classified into 
proficiency levels. There are two kinds of classification reliability statistics: decision accuracy and 
decision consistency. Decision accuracy is the agreement between the classifications actually made and 
the classifications that would be made if the test scores were perfectly reliable. Decision consistency is 
the agreement between the classifications that would be made on two independent different forms of 
the test. 


Another index is inter-rater reliability for the human-scored constructed-response items, which 
measures the agreement between individual raters (scorers). The inter-rater reliability coefficient 
answers the question, “How consistent would the scores of these test takers be over replication of 
scoring of the same responses by different scorers?” 


Standard error of measurement (SEM) quantifies the amount of error in the test scores. SEM is the 
extent by which test takers’ scores tend to differ from the scores they would receive if the test were 
perfectly reliable. As the SEM increases, the variability of students’ observed scores is likely to increase 
across repeated testing. Observed scores with large SEMs pose a challenge to the valid interpretation of 
a single test score. 


March 27, 2018 Page 93 


PARCC 2017 Technical Report 


Reliability and SEM estimates were calculated at the full assessment level—both paper-based tests 
(PBTs) and computer-based tests (CBTs)—and at the claim and subclaim levels. In addition, conditional 
SEMs were calculated and reported in Section 12 and Appendix 12.4. 


8.2 Reliability and SEM Estimation 
8.2.1 Raw Score Reliability Estimation 


Coefficient alpha (Cronbach, 1951), which measures internal consistency reliability, is the most 
commonly used measure of reliability. Coefficient alpha is estimated by substituting sample estimates 
for the parameters in the formula below: 


, (8-1) 


where n_ is the number of items, o . is the variance of scores on the i-th item, and o . is the variance 


of the total score (sum of scores on the individual items). Other things being equal, the more items a 
test includes, the higher the internal consistency reliability. 


Since PARCC test forms have mixed item types (dichotomous and polytomous items), it is more 
appropriate to report stratified alpha (Feldt & Brennan, 1989). Stratified alpha is a weighted average of 
coefficient alphas for item sets with different maximum score points or “strata.” Stratified alpha is a 
reliability estimate computed by dividing the test into parts (strata), computing alpha separately for 
each part, and using the results to estimate a reliability coefficient for the total score. Stratified alpha is 
used here because different parts of the test consist of different item types and may measure different 
skills. The formula for the stratified alpha is: 


Das 9%; (1-a)) 


Pstrata = 1 2 
oX 


(8-2) 


where Oo, is the variance for part j of the test, o s is the variance of the total scores, and a , is 


coefficient alpha for part j of the test. Estimates of stratified alpha are computed by substituting sample 
estimates for the parameters in the formula. The average stratified alpha is a weighted average of the 
stratified alphas across the test forms. 


The formula for the standard error of measurement is: 


OF = Oy 1— pxx, (8-3) 


where Oy is the standard deviation of the test raw score and pyy, is the reliability estimated by 
substitution of appropriate statistics for the parameters in equation 8-1 or 8-2. 
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8.2.2 Scale Score Reliability Estimation 


Like the stratified alpha coefficients given in section 8, scale score reliability coefficients range from 0 to 
1. The higher the reliability coefficient for a set of scores, the more likely individuals would be to obtain 
similar scores upon repeated testing occasions, if the students do not change in their level of the 
knowledge or skills measured by the test. Because PARCC scale scores are computed from a total score 
and do not have an item-level component, a stratified alpha coefficient cannot be computed for scale 
scores. Instead, Kolen, Zeng, and Hanson’s (1996) method for scale score reliability was used. 


The general formula for a reliability coefficient, 


o*(E) 


p= a o2(xy (8-1) 


involves the error variance, o7(E), and the total score variance, 0*(X). Using Kolen et al.’s (1996) 
method, conditional raw score distributions are estimated using Lord and Wingersky’s (1984) recursion 
formula. The conditional raw score distributions are transformed into conditional scale score 
distributions. Denote x as the raw sum score ranging from 0 to X, and s as a resulting scale score after 
transformation. The conditional distribution of scale scores is written as P(X = x|@). The mean and 
variance, o7[s(X)], of this distribution can be computed using these scores and their associated 


probabilities. 
The average error variance of the scale scores is computed as 
o* (Errorscale) = d, o*(s(X)|8) g(@)dé, (8-2) 


where g(@) is the ability distribution. The square root of the error variance is the conditional standard 


error of measurement of the scale scores. 


Just as the reliability of raw scores is one minus the ratio of error variance to total variance, the 
reliability of scale scores is one minus the ratio of the average variance of measurement error for scale 
scores to the total variance of scale scores, 

o* (Error scale) 


Pscale = ioe 215(Xx 
o*[s(X)] (8-3) 


The Windows program POLYCSEM (Kolen, 2004) was used to estimate scale score error variance and 


reliability. 
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8.3 Reliability Results for Total Group 
8.3.1 Raw Score Reliability Results 


Tables 8.1 and Table 8.2 summarize test reliability estimates for the total testing group for English 
language arts/literacy (ELA/L) and mathematics, respectively. The section includes only spring 2017 
results. The fall 2016 results are located in the Addendum.’ The tables provide the average reliability, 
which is estimated by averaging the internal consistency estimates computed for all the individual forms 
of the test and the raw score SEMs, separately for the CBTs and PBTs within each grade level. In 
addition, the number of forms, the total sample size across all forms, and the average maximum 
possible score for each set of tests are provided. Estimates were calculated only for groups of 100 or 
more students administered a specific test form. 


English Language Arts/Literacy 

The average reliability estimates for the CBT tests for grades 3 through 11 ELA/L range from a low of .91 
to a high of .94. The average reliability estimates for the PBT tests for ELA/L grades 3 through 11 range 
from a low of .86 to a high of .94. The tests for grades 3 through 5 have fewer maximum possible points 
than for the grades 6 through 11 tests. The average reliability estimates are at least .90 except for 
grades 4 and 5 PBT tests, which are .88, and grade 11 PBT, which is .86. The average raw score SEM is 
consistently between 5 percent to 6 percent of the maximum possible score. 


5 Addendum 8 provides a summary of reliability information for the fall 2016 administration. 
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Table 8.1 Summary of ELA/L Test Reliability Estimates for Total Group 


Avg. Max. Min. Max. 
Grade ___ Testing Number Possible Avg. Raw Avg. Sample Min. Sample Max. 
Level Mode of Forms Score Score SEM __ Reliability Size Reliability Size Reliability 

CBT 4 94 4.93 0.92 8,088 0.86 114,767 0.92 

: PBT 2 94 5.39 0.90 2,199 0.85 53,719 0.91 
CBT 5 106 5.65 0.91 9,281 0.88 159 0.92 

‘ PBT 2 106 6.02 0.88 2,333 0.84 45,527 0.89 
CBT 5 105 5.47 0.91 9,562 0.84 128,027 0.92 

2 PBT 2 106 5.97 0.88 1,068 0.86 13,636 0.88 
CBT 5 121 6.08 0.93 9,161 0.87 124,574 0.94 

PBT 2 121 6.65 0.92 895 0.90 161,54 0.92 
CBT 5 121 6.23 0.93 157 0.90 125,944 0.94 

: PBT 2 121 6.85 0.91 695 0.88 14,688 0.91 
CBT 5 121 6.19 0.93 7,591 0.89 125,202 0.94 

: PBT 2 121 6.85 0.91 660 0.90 13,287 0.92 
CBT 4 121 6.03 0.94 3,044 0.88 60,189 0.94 

2 PBT 2 121 6.35 0.93 2,791 0.93 106 0.93 
CBT 5 121 6.24 0.93 2,414 0.88 54,430 0.94 

0 PBT 1 121 6.45 0.94 1,740 0.94 1,740 0.94 
CBT 4 121 6.08 0.93 1,604 0.88 34,137 0.93 

= PBT 1 121 6.65 0.86 1,069 0.86 1,069 0.86 


Mathematics 

The average reliability estimates for the grades 3 through 8 mathematics assessments range from .90 to 
.94 for the CBT tests and from .86 to .93 for the PBT tests. Most of the average reliability estimates are 
above .90 except for two PBT and one CBT. The raw score SEM consistently ranges from 4 percent to 6 
percent of the maximum score. 
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Table 8.2 Summary of Mathematics Test Reliability Estimates for Total Group 


Avg. Max. Min. Max. 
Grade __ Testing Number Possible Avg. Raw Avg. Sample Min. Sample Max. 
Level Mode of Forms Score Score SEM _ Reliability Size Reliability Size Reliability 
CBT 6 66 3.53 0.94 2,007 0.92 34,750 0.94 
2 PBT 2 66 3.75 0.93 5,670 0.90 50,959 0.93 
CBT 6 66 3.49 0.94 1,966 0.92 102,282 0.94 
2 PBT 2 66 3.43 0.93 5,065 0.88 42,538 0.93 
CBT 6 66 3.48 0.93 1,784 0.89 46,338 0.93 
2 PBT 2 66 3.41 0.92 2,176 0.87 13,450 0.92 
CBT 7 66 3.44 0.94 107 0.84 108,745 0.94 
° PBT 2 66 3.60 0.92 1,603 0.84 15,276 0.93 
CBT 8 66 3.43 0.92 108 0.70 32,369 0.93 
? PBT 2 66 3.41 0.89 1,253 0.79 12,199 0.90 
CBT 6 66 3.26 0.91 884 0.90 83,333 0.92 
- PBT 2 66 3.51 0.89 1,173 0.75 11,058 0.90 
CBT 6 81 3.61 0.94 488 0.85 67,566 0.94 
a PBT 2 81 3.73 0.92 406 0.86 5,623 0.92 
ea CBT 6 80 3.48 0.94 204 0.86 35,202 0.95 
PBT 1 81 3.6 0.93 1,306 0.93 1,306 0.93 
CBT 5 81 3.51 0.93 4,818 0.92 35,156 0.93 
a PBT 1 81 4.01 0.93 2,051 0.93 2,051 0.93 
CBT 2 81 3.53 0.92 425 0.71 9,376 0.93 
PBT 
CBT 1 80 3.73 0.90 1,914 0.90 1,914 0.90 
ie PBT 1 80 3.21 0.86 182 0.86 182 0.86 
‘ie CBT 1 81 3.89 0.93 477 0.93 477 0.93 
PBT 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics II, M3 = Integrated Mathematics III. 


8.3.2 Scale Score Reliability Results 


Tables 8.3 and Table 8.4 summarize scale score reliability estimates for the total testing group for 
English language arts/literacy (ELA/L) and mathematics, respectively, for spring 2017. The tables provide 
average reliabilities by grade level and mode, which are estimated by averaging the reliability estimates 
computed for all forms of the test within mode and grade level. In addition, the number of forms, the 
total sample size across all forms, and the average maximum possible score for each set of tests are 
provided. Since estimates of scale score reliability are sample independent, all of the form-level results 
are included. 
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English Language Arts/Literacy 

The average reliability estimates for CBTs for grades 3-11 English language arts/literacy (ELA/L) range 
from .902 to .932. The average reliability estimates for the PBTs for ELA/L grades 3-11 range from .899 
to .936. The average reliability estimates are at least .899. The average scale score SEM is consistently 
between 8.472 to 11.583 for CBTs and between 8.628 and 11.967 for PBTs. 


Mathematics 

The average scale score reliability estimates for the grades 3-8 mathematics assessments range from 
.875 to .928 for the CBTs and from .856 to .929 for the PBTs. For the high school assessments, these 
quantities range from .839 to .892 and from .848 and .898 for CBTs and PBTs, respectively. For grades 3- 
8, the scale score SEM consistently ranges from 8.449 to 12.436 for CBTs and from 8.391 to 13.462 for 
PBTs. For high school tests, the scale score SEM ranges from 9.278 to 13.818 for CBTs and from 8.952 to 
14.027 for PBTs. 


Table 8.3 Summary of ELA/L Test Scale Score Reliability Estimates for Total Group 


Testing Number of Avg. Scale Avg. Scale Min. Scale Max. Scale 
aaa Mode Forms Score SEM Score Score Score 
Reliability Reliability Reliability 
3 CBT 5 11.583 0.909 0.902 0.921 
PBT 2 11.967 0.904 0.904 0.904 
4 CBT 5 9.838 0.913 0.904 0.923 
PBT 2 9.829 0.913 0.912 0.914 
5 CBT 6 9.881 0.902 0.892 0.915 
PBT 2 9.745 0.904 0.894 0.915 
6 CBT 6 8.472 0.921 0.914 0.925 
PBT 2 8.628 0.918 0.917 0.919 
7 CBT 6 9.798 0.924 0.916 0.932 
PBT 2 10.412 0.915 0.913 0.918 
8 CBT 6 9.636 0.927 0.921 0.932 
PBT 2 10.118 0.920 0.918 0.923 
9 CBT 5 9.710 0.927 0.917 0.934 
PBT 2 10.169 0.920 0.919 0.921 
10 CBT 5 11.408 0.932 0.929 0.938 
PBT 2 11.061 0.936 0.935 0.938 
11 CBT 5 10.869 0.913 0.903 0.919 
PBT 2 11.766 0.899 0.895 0.904 


March 27, 2018 Page 99 


PARCC 2017 Technical Report 


Table 8.4 Summary of Mathematics Test Scale Score Reliability Estimates for Total Group 
Avg. Scale Min. Scale Max. Scale 


Testing Number of Avg. Scale 
Grade Level Mode Forms Score SEM peers - penne 7 ore a 
Reliability Reliability Reliability 
3 CBT 7 9.066 0.928 0.922 0.932 
PBT 2 9.287 0.925 0.924 0.926 
4 CBT 7 8.449 0.928 0.923 0.931 
PBT 2 8.391 0.929 0.929 0.930 
5 CBT 7 8.667 0.921 0.916 0.926 
PBT 2 8.547 0.922 0.922 0.923 
6 CBT 7 8.702 0.916 0.910 0.922 
PBT 2 8.739 0.915 0.915 0.915 
7 CBT 7 9.006 0.892 0.880 0.898 
PBT 2 9.337 0.885 0.882 0.887 
8 CBT 6 12.436 0.875 0.855 0.882 
PBT 2 13.462 0.856 0.847 0.865 
Al CBT 6 11.795 0.882 0.875 0.887 
PBT 2 11.301 0.888 0.888 0.888 
G1 CBT 7 9.278 0.892 0.877 0.901 
PBT 2 8.952 0.898 0.895 0.900 
A2 CBT 7 13.117 0.887 0.881 0.893 
PBT 2 13.137 0.886 0.885 0.888 
M1 CBT 5 11.820 0.881 0.861 0.906 
PBT 2 11.938 0.879 0.879 0.880 
M2 CBT 3 12.857 0.839 0.830 0.846 
PBT 2 12.476 0.848 0.846 0.850 
M3 CBT 3 13.818 0.875 0.865 0.881 
PBT 2 14.027 0.872 0.865 0.878 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics II, M3 = Integrated Mathematics III. 
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8.4 Reliability Results for Subgroups of Interest 


When the sample size was sufficiently large, raw score reliability and SEM were estimated for the groups 
identified for DIF analysis. Estimates were calculated only for groups of 100 or more students 
administered a specific test form. 


Tables 8.5 through 8.13 summarize test reliability for groups of interest for ELA/L grades 3 through 11, 
and Tables 8.14 through 8.25 summarize test reliability for groups of interest for mathematics 
grades/subjects. Note that reliability estimates are dependent on score variance, and subgroups with 
smaller variance are likely to have lower reliability estimates than the total group. 


Gender 

English Language Arts/Literacy 

The average reliability estimates and the average SEMs for males and females are similar to the 
corresponding values for the total group. For most tests, the reliabilities between males and females are 
equal; for three tests, the reliability for males was .01 higher than for females. The SEMs for females are 
all higher than for males. 


Mathematics 

As with the ELA/L test components, the average reliability estimates and SEMs for males and females 
reflect the corresponding reliabilities for the total group. Typically, the reliabilities are a little higher for 
males than for females. The SEMs are generally very similar for females and males. 


Ethnicity 

English Language Arts/Literacy 

The majority of the reliabilities for the ethnicity groups are .01 to .02 lower than for the total group. 
There is not a consistent difference among the test reliabilities for white, African American, Asian/Pacific 
Islander, Hispanic, and multiple-ethnicity students, with the majority of the reliabilities between .89 and 
.94. However, the majority of the reliabilities for American Indian/Alaskan native students range from 
.88 to .92. In general, the SEMs are similar to the total group SEMs. For most grade levels, the SEMs are 
highest for Asian/Pacific Islander students. 


Mathematics 

As with the ELA/L reliabilities, the reliabilities for ethnicity groups are marginally lower than for the total 
group of students. The average SEMs reflect the total group SEMs. While there is variation across tests, 
the average reliabilities are generally highest for Asian/Pacific Islander students. The American 
Indian/Alaskan native and the African American groups have the lowest reliabilities. 
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Special Education Needs 

English Language Arts/Literacy 

The reliabilities for five groups of students (economically disadvantaged, not economically 
disadvantaged, non-English learner, students with disabilities, and students without disabilities) are 
generally .01 to .02 less than those for the total group of students. The majority of the reliabilities range 
from .88 to .93. The average reliabilities for English learner students are lower, ranging from .84 to .88. 
The SEMs are generally similar to the total group SEMs. 


Mathematics 

The average reliabilities for the larger student groups (not economically disadvantaged, non-English 
learner, and students without disabilities) are quite similar to the total group of students. For 
economically disadvantaged, English learner, and students with disabilities, the average reliabilities are 
lower than those for the total group. In general, the SEMs are similar to the total group SEMs. 


Students Taking Accommodated Forms 

English Language Arts/Literacy 

Two of the four accommodation form types had sufficient sample sizes to allow for estimation of 
reliability and SEM except grades 10 and 11, which had three and one of the four forms that had a 
sufficient sample, respectively. In the lower grades, the ASL and screen reader groups did not have at 
least 100 students take any specific form. Within grades, the reliabilities of the closed caption forms are 
similar to the average reliabilities for the total group. For the text-to-speech forms, the reliabilities are 
somewhat lower than for the total group. 


Mathematics 

The text-to-speech forms had sufficient sample sizes for reliability and SEM estimation across grades, 
except for the Integrated Mathematics II and III courses where the sample was not sufficient. ASL forms 
had more than 100 students for grade 6 through 8 and Algebra 1. For almost all tests, text-to-speech 
reliabilities and SEMs are very close to the total group reliabilities. For grade 6, the ASL reliability was 
.84, but for the other grades these reliabilities were lower. The SEMs for accommodated forms were 
similar to the total group. 


Students Taking Translated Forms 

Mathematics 

With the exception of Integrated Mathematics |, II, and Ill, there were sufficient numbers of students 
taking the Spanish-language form for reliability and SEM estimation. The average reliability ranged from 
.68 to .91. The SEMs are generally lower for the students administered the Spanish-language forms. The 
moderate to high correlations suggest the translated forms are sufficient for individual student 
reporting. 
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Table 8.5 Summary of Test Reliability Estimates for Subgroups: Grade 3 ELA/L 


CBT PBT 
Max. Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max 
Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample _ Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 94 4.93 0.92 8,088 0.86 114,767 0.92 94 5.39 0.90 2,199 0.85 53,719 0.91 
Gender 
Male 94 4.82 0.92 5,239 0.85 58,237 0.92 94 5.22 0.90 1,438 0.84 26,759 0.91 
Female 94 5.05 0.91 2,849 0.86 56,530 0.92 94 5.56 0.90 761 0.86 26,960 0.91 
Unknown/Missing . - - - - = . - 7 7 7 : . 7 
Ethnicity 
White 94 5.06 0.90 3,014 0.86 42,290 0.91 94 85.54 0.90 445 0.86 14,949 0.90 
African American 94 =4.73 0.90 1,999 0.8 17,806 0.92 94 =5.18 0.89 786 0.85 13,692 0.89 
Asian/Pacific Islander 94 5.09 0.91 17,411 09 17,562 0.92 94 5.61 0.90 4714 0.90 4,714 0.90 
Am. Indian/Alaska Native 94 4.56 0.89 1,418 0.88 1,418 0.90 94 85.23 0.85 1,394 0.85 1,394 0.85 
Hispanic 94 4.77 0.90 2,536 0.83 31,352 0.92 94 «5.36 0.89 861 0.83 17,370 0.89 
Multiple 94 4,93 0.92 226 0.87 4,236 0.92 94 =5.50 0.90 1,571 0.90 1,571 0.90 
Special Instructional Needs 
Economically 
Disadvantaged 94 4.72 0.90 4,750 0.80 53,596 0.91 94 85.26 0.88 1,858 0.83 34,427 0.89 
Not Economically 
Disadvantaged 94 5,11 0.90 3,332 0.88 60,982 0.91 94 5.61 0.89 336 0.89 1,274 0.89 
English Learner 94 §=4.48 0.88 1,011 O.75 15,273 0.89 94 85.23 0.87 599 0.78 11,816 0.88 
Non-English Learner 94 5.00 0.91 7,058 0.86 96,572 0.92 94 85.43 0.91 1,593 0.86 41,809 0.91 
Students with Disabilities 94 4,37 0.91 8,088 0.86 14,480 0.93 94 4.57 0.89 2,138 085 5,302 0.90 
Students w/o Disabilities 94 5.02 0.91 99,376 0.90 99,873 0.92 94 5.51 0.90 48,303 0.90 48,303 0.90 
Students Taking 
Accommodated Forms 
A: ASL - - : - 7 7 : - : 7 
C: Closed Caption 94 4.85 0.93 244 0.93 244 0.93 - - - - - - - 
R: Screen Reader - - : - - 7 . 7 - - - 7 _ 
T: Text-to-Speech 94 4.20 0.85 7,844 0.85 7,844 0.85 - - - - - - - 
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CBT PBT 
Max. Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max Max. 
Raw Avg. Relia- Sample’ Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 106 5.65 0.91 9,281 0.88 159 0.92 106 6.02 0.88 2,333 0.84 45,527 0.89 
Gender 
Male 106 5.49 0.91 5,942 0.87 103 0.94 106 5.80 0.88 1,508 0.84 22,661 0.89 
Female 106 5.81 0.91 3,339 0.88 58,657 0.92 106 6.22 0.88 825 0.84 22,866 0.88 
Unknown/Missing = - - = 7 7 . 7 . - - - - - 
Ethnicity 
White 106 5.78 0.90 3,475 0.88 44,526 0.91 106 6.17 0.88 392 0.86 11,355 0.88 
African American 106 §=5.39 0.90 2,315 0.84 18,220 0.91 106 5.82 0.85 844 0.82 11,529 0.86 
Asian/Pacific Islander 106 5.91 0.91 17,847 0.90 246 0.92 106 6.22 0.88 3,972 0.88 3,972 0.88 
Am. Indian/Alaska Native 106 5.05 0.89 1,412 0.88 1,422 0.90 106 5.79 0.83 1,313 0.83 1,313 0.83 
Hispanic 106 5.45 0.90 2,927 0.86 32,806 0.91 106 5.97 0.86 1,002 0.82 16,237 0.86 
Multiple 106 5.66 0.91 236 0.87 4,313 0.92 106 6.15 0.89 1,062 0.89 1,062 0.89 
Special Instructional Needs 
Economically 
Disadvantaged 106 5.36 0.89 5,463 0.84 54,981 0.91 106 5.89 0.86 1,969 0.82 30,561 0.86 
Not Economically 
Disadvantaged 106 5.86 0.90 3,816 0.88 63,961 0.91 106 6.23 0.87 364 0.87. 14,915 0.87 
English Learner 106 4.76 0.85 938 0.77. 10,193 0.87 106 5.43 0.80 672 0.79 6,452 0.80 
Non-English Learner 106 5.72 0.91 8,311 0.88 135 0.93 106 6.11 0.88 1,656 0.85 38,962 0.88 
Students with Disabilities 106 4.93 0.91 9,281 0.88 15,872 0.93 106 $5.15 0.88 2,291 0.84 5,149 0.89 
Students w/o Disabilities 106 5.77 0.90 103,959 0.90 102,906 0.91 106 6.16 0.87 40,246 0.87 40,246 0.87 
Students Taking 
Accommodated Forms 
A: ASL : = - : = : : = 7 : - 7 
C: Closed Caption 106 5.64 0.93 236 0.93 236 0.93 - - - - - - - 
R: Screen Reader 7 7 ‘ - - - - - - . - - - 7 
T: Text-to-Speech 106 4.99 0.87 9,045 0.87 9,045 0.87 - - - - - - - 
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CBT PBT 
Max. Avg. Min. Min. Max. Max. Max Avg. Min. Min. Max Max. 
Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 105 5.47 0.91 9,562 0.84 128,027 0.92 106 5.97 0.88 1,068 0.86 13,636 0.88 
Gender 
Male 105 5.28 0.91 6,100 0.83 64,869 0.92 106 =5.73 0.88 675 0.85 7,022 0.88 
Female 105 5.65 0.91 3,462 0.84 63,158 0.92 106 6.23 0.87 6,614 0.87 393 0.88 
Unknown/Missing - - - = 7 - . 7 ° - . - - - 
Ethnicity 
White 105 5.64 0.90 3,601 0.85 46,692 0.91 106 6.17 0.86 5,057 0.86 224 0.88 
African American 105 =5.14 0.89 2,395 0.75 20,946 0.91 106 5.44 0.86 385 0.82 1,795 0.87 
Asian/Pacific Islander 105 «(5.71 0.91 18,416 0.90 18,305 0.91 106 6.27 0.87 2,121 0.87 2,121 0.87 
Am. Indian/Alaska Native 105 5.05 0.88 1,317 0.88 1,463 0.89 106 5.85 0.79 1,176 0.79 1,176 0.79 
Hispanic 105 5.29 0.89 3,002 0.81 36,109 0.91 106 5.81 0.85 387 0.77 3,016 0.86 
Multiple 105 5.48 0.91 205 0.81 4,253 0.92 106 6.26 0.88 442 0.88 442 0.88 
Special Instructional Needs 
Economically 
Disadvantaged 105 5.20 0.89 5,585 0.78 61,077 0.90 106 5.74 0.85 863 0.79 7,080 0.86 
Not Economically 
Disadvantaged 105 5.69 0.90 3,971 0.86 66,738 0.91 106 §6.23 0.86 6,525 0.86 204 0.91 
English Learner 105 4.43 0.83 917 0.59 8,697 0.86 106 5.51 0.76 228 0.64 1,446 0.77 
Non-English Learner 105 5.53 0.90 8,615 0.84 116,218 0.91 106 6.02 0.88 833 0.87 12,097 0.88 
Students with Disabilities 105 4.72 0.90 9,562 0.84 17,677 0.92 106 5.00 0.87 1,048 O86 2,404 0.87 
Students w/o Disabilities 105 5.60 0.90 110,069 0.89 109,924 0.91 106 6.26 0.86 11,177. 0.86 11,177 0.86 
Students Taking 
Accommodated Forms 
A: ASL : = ? : 7 : = 7 : - 7 
C: Closed Caption 106 5.30 0.90 303 0.90 303 0.90 - - - - - - - 
R: Screen Reader - - z 7 - - - - - . - - - 7 
T: Text-to-Speech 106 4.47 0.83 9,259 0.83 9,259 0.83 - - - - - - - 
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CBT PBT 
Max. Avg. Min. Min. Max. Max. Max Avg. Min Min. Max Max. 
Raw Avg. _ Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample  Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 121 6.08 0.93 9,161 0.87. 124,574 0.94 121 6.65 0.92 895 0.90 161,54 0.92 
Gender 
Male 121 5.87 0.93 5,846 0.87. 64,165 0.94 121 6.41 0.92 570 0.88 8,262 0.92 
Female 121 6.28 0.93 3,315 0.88 61,320 0.93 121 6.90 0.91 325 0.91 7,892 0.91 
Unknown/Missing 7 : : . : : ci - ss : 
Ethnicity 
White 121 6.23 0.93 3,653 0.89 46,483 0.93 121 6.67 0.91 222 0.90 6,939 0.91 
African American 121 5.78 0.92 2,263 0.81 20,111 0.92 121 6.31 0.91 322 0.86 2,407 0.91 
Asian/Pacific Islander 121 6.30 0.93 270 0.93 17,571 0.94 121 6.92 0.90 2,286 0.90 2,286 0.90 
Am. Indian/Alaska Native 121 5.68 0.92 1,277 0.91 1,210 0.92 121 6.94 0.87 1,122 0.87 1,122 0.87 
Hispanic 121 5.91 0.92 2,687 0.85 36,029 0.92 121 6.52 0.91 290 0.87 2,792 0.91 
Multiple 121 6.10 0.94 205 0.83 3,857 0.94 121 6.66 0.92 592 0.92 592 0.92 
Special Instructional Needs 
Economically 
Disadvantaged 121 5.84 0.92 5,226 0.82 58,773 0.92 121 6.52 0.90 706 0.85 7,462 0.91 
Not Economically 
Disadvantaged 121 6.27 0.93 3,925 0.89 66,107 0.93 121 6.76 0.91 8,678 0.91 187 0.94 
English Learner 121 4.91 0.85 753 0.62 7,349 0.87 121 6.11 0.86 180 0.86 1,153 0.86 
Non-English Learner 121 6.14 0.93 8,386 0.88 114,491 0.93 121 6.68 0.92 708 0.90 14,914 0.92 
Students with Disabilities 121 5.28 0.92 9,161 0.87. 17,932 0.93 121 5.79 0.91 869 0.90 2,623 0.91 
Students w/o Disabilities 121 6.22 0.93 107,452 0.93 106,634 0.93 121 6.86 0.90 13,492 0.90 13,492 0.90 
Students Taking 
Accommodated Forms 
A: ASL 7 - : = . 7 - = - ss 
C: Closed Caption 121 5.61 0.95 298 0.95 298 0.95 - - - - - - - 
R: Screen Reader - 7 - - - - - - - - - z - . 
T: Text-to-Speech 121 5.05 0.86 8,863 0.86 8,863 0.86 - - - - - - - 
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CBT PBT 
Max. Avg. Min. Min. Max. Max. Max Avg. Min Min. Max Max. 
Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 121 6.23 0.93 157 0.90 125,944 0.94 121 6.85 0.91 695 0.88 14,688 0.91 
Gender 
Male 121 6.03 0.93 5,443 0.89 63,855 0.94 121 6.58 0.91 430 0.83 7,442 0.91 
Female 121 6.43 0.93 2,860 0.90 62,089 0.93 121 7.10 0.90 7,246 0.90 265 0.90 
Unknown/Missing = - - = 7 - - 7 ° - - - - 7 
Ethnicity 
White 121 6.35 0.92 3,405 0.91 48,125 0.93 121 6.93 0.91 6,512 0.91 161 0.92 
African American 121 5.99 0.92 1,965 0.84 20,086 0.93 121 6.46 0.90 251 0.84 2,600 0.90 
Asian/Pacific Islander 121 6.42 0.93 242 0.93 17,517 0.94 121 7.15 0.90 1,862 0.90 1,862 0.90 
Am. Indian/Alaska Native 121 5.97 0.91 1,346 0.91 1,360 0.92 121 6.82 0.85 827 0.85 827 0.85 
Hispanic 121 6.09 0.92 2,488 0.86 34,982 0.93 121 6.73 0.88 238 0.79 2,314 0.89 
Multiple 121 6.28 0.93 148 0.90 3,692 0.94 121 7.05 0.90 559 0.90 559 0.90 
Special Instructional Needs 
Economically 
Disadvantaged 121 6.03 0.92 4,418 0.85 56,926 0.93 121 6.69 0.88 521 0.81 6,560 0.89 
Not Economically 
Disadvantaged 121 6.38 0.93 3,873 0.91 68,806 0.93 121 6.97 0.90 8,114 0.90 174 0.93 
English Learner 121 5.19 0.86 712 0.73 7,082 0.88 121 6.28 0.80 132 0.65 876 0.83 
Non-English Learner 121 6.28 0.93 120 0.90 116,025 0.94 121 6.88 0.91 562 0.89 13,722 0.91 
Students with Disabilities 121 5.52 0.92 157 0.90 17,506 0.93 121 5.92 0.89 684 0.88 2,403 0.89 
Students w/o Disabilities 121 6.36 0.93 108,772 0.92 107,918 0.93 121 7.06 0.89 12,251 089 12,251 0.89 
Students Taking 
Accommodated Forms 
A: ASL . = 7 - = 7 : = 7 : - 7 
C: Closed Caption 121 5.96 0.94 363 0.94 363 0.94 - - - - - - - 
R: Screen Reader = - 7 - - : - - - . - - - = 
T: Text-to-Speech 121 5.20 0.89 7,940 0.89 7,940 0.89 - - - - - - - 
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Table 8.10 Summary of Test Reliability Estimates for Subgroups: Grade 8 ELA/L 


CBT PBT 
Max. Avg. Min. Min. Max. Max. Max Avg. Min Min. Max Max. 
Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 121 6.19 0.93 7,591 0.89 125,202 0.94 121 6.85 0.91 660 0.90 13,287 0.92 
Gender 
Male 121 6.02 0.93 4,968 0.88 63,844 0.94 121 6.58 0.92 417 0.89 6,862 0.92 
Female 121 6.35 0.93 2,623 0.89 61,507 0.93 121 7.11 0.90 6,425 0.90 243 0.90 
Unknown/Missing = - - 7 = - - 7 7 - - - - - 
Ethnicity 
White 121 6.30 0.93 3,110 0.90 48,592 0.93 121 6.85 0.91 5,527. 0.91 183 0.93 
African American 121 5.97 0.92 1,921 0.85 20,143 0.93 121 6.50 0.90 193 0.87 2,290 0.91 
Asian/Pacific Islander 121 6.33 0.93 224 0.92 17,050 0.94 121 7.03 0.91 1,643 0.91 1,643 0.91 
Am. Indian/Alaska Native 121 5.87 0.92 1,249 0.91 1,221 0.92 121 7.22 0.85 791 0.85 791 0.85 
Hispanic 121 6.06 0.92 2,151 0.87 34,312 0.93 121 6.88 0.90 234 0.86 2,598 0.90 
Multiple 121 6.16 0.94 149 0.90 3,537 0.94 121 6.78 0.92 422 0.92 422 0.92 
Special Instructional Needs 
Economically 
Disadvantaged 121 5.99 0.92 4,007 0.86 55,736 0.92 121 6.85 0.89 472 0.86 6,453 0.90 
Not Economically 
Disadvantaged 121 6.33 0.93 3,565 0.90 69,257 0.93 121 6.84 0.91 6,822 0.91 187 0.93 
English Learner 121 5.19 0.87 531 0.79 6,964 0.87 121 6.82 0.82 131 0.78 983 0.83 
Non-English Learner 121 6.23 0.93 7,028 0.89 115,665 0.93 121 6.85 0.91 526 0.91 12,246 0.91 
Students with Disabilities 121 5.55 0.92 7,591 0.89 17,371 0.93 121 6.18 0.91 642 0.90 2,247 0.91 
Students w/o Disabilities 121 6.29 0.93 106,507 0.92 107,005 0.93 121 7.01 0.90 11,014 0.90 11,014 0.90 
Students Taking 
Accommodated Forms 
A: ASL . . 7 = = . : = 7 : - 7 
C: Closed Caption 121 6.07 0.94 312 0.94 312 0.94 - - - - - - - 
R: Screen Reader = - 7 - - 7 . - - . - - - = 
T: Text-to-Speech 121 5.36 0.88 7,279 0.88 7,279 0.88 - - - - - - - 
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Table 8.11 Summary of Test Reliability Estimates for Subgroups: Grade 9 ELA/L 


CBT PBT 
Max. Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 121 6.03 0.94 3,044 0.88 60,189 0.94 121 6.35 0.93 2,791 0.93 106 0.93 
Gender 
Male 121 5.83 0.94 1,980 0.88 30,989 0.94 121 5.95 0.93 1,526 0.93 1,526 0.93 
Female 121 6.21 0.94 1,064 0.88 29,200 0.94 121 6.82 0.92 1,265 0.92 1,265 0.92 
Unknown/Missing = - - = 7 - - - 7 - - - . - 
Ethnicity 
White 121 6.24 0.93 1,208 0.88 19,286 0.93 121 6.24 0.93 324 0.93 324 0.93 
African American 121 5.84 0.92 704 0.84 6,535 0.93 121 5.76 0.91 166 0.91 166 0.91 
Asian/Pacific Islander 121 6.18 0.94 124 0.91 12,125 0.94 121 6.73 0.93 788 0.93 788 0.93 
Am. Indian/Alaska Native 121 5.61 0.90 1,091 0.89 1,000 0.91 121 6.24 0.86 186 0.86 186 0.86 
Hispanic 121 5.77 0.93 903 0.87. 19,811 0.93 121 6.23 0.90 1,246 0.90 1,246 0.90 
Multiple 121 6.13 0.94 1,219 0.94 1,275 0.94 - - - - - - - 
Special Instructional Needs 
Economically 
Disadvantaged 121 5.75 0.92 1,410 0.84 24,725 0.93 121 6.16 0.90 1,551 0.90 1,551 0.90 
Not Economically 
Disadvantaged 121 6.20 0.93 1,620 0.89 35,340 0.94 121 6.62 0.93 1,215 0.93 1,215 0.93 
English Learner 121 4.78 0.86 104 0.71 4,858 0.87 121 5.77 0.85 521 0.85 521 0.85 
Non-English Learner 121 6.12 0.93 2,920 0.88 53,301 0.94 121 6.47 0.93 2,216 0.93 2,216 0.93 
Students with Disabilities 121 5.36 0.92 3,044 0.88 8,747 0.93 121 5.60 0.92 530 0.92 530 0.92 
Students w/o Disabilities 121 6.14 0.94 51,078 0.93 51,248 0.94 121 6.53 0.92 2,235 0.92 2,235 0.92 
Students Taking 
Accommodated Forms 
A: ASL . : - = - : = 7 : - 7 
C: Closed Caption 121 6.38 0.93 111 0.93 111 0.93 - - - - - - - 
R: Screen Reader 7 - ‘ - - - - - - . - . - = 
T: Text-to-Speech 121 5.22 0.87 2,933 0.87 2,933 0.87 - - - - - - - 
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Table 8.12 Summary of Test Reliability Estimates for Subgroups: Grade 10 ELA/L 


CBT PBT 
Max. Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Raw Avg. _ Relia- Sample _ Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample _ Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 121 6.24 0.93 2,414 0.88 54,430 0.94 121 6.45 0.94 1,740 0.94 1,740 0.94 
Gender 
Male 121 6.02 0.94 1,584 0.87. 927,572 0.94 121 6.18 0.94 929 0.94 929 0.94 
Female 121 6.45 0.93 830 0.89 26,858 0.93 121 6.73 0.94 811 0.94 811 0.94 
Unknown/Missing = - - . = - - - . - - - - 7 
Ethnicity 
White 121 6.44 0.92 1,016 0.90 22,014 0.93 121 6.64 0.94 847 0.94 847 0.94 
African American 121 5.95 0.92 589 0.83 11,374 0.93 121 6.13 0.94 346 0.94 346 0.94 
Asian/Pacific Islander 121 6.61 0.92 4,349 0.92 4,445 0.93 - - - - - - - 
Am. Indian/Alaska Native 121 5.83 0.91 804 0.90 845 0.91 121 6.32 0.85 196 0.85 196 0.85 
Hispanic 121 6.01 0.92 617 0.84 14,627 0.93 121 6.16 0.94 220 0.94 220 0.94 
Multiple 121 6.29 0.93 1,171 0.93 1,108 0.93 - - - - - - - 
Special Instructional Needs 
Economically 
Disadvantaged 121 5.93 0.92 1,098 0.84 20,834 0.93 121 6.32 0.92 701 0.92 701 0.92 
Not Economically 
Disadvantaged 121 6.41 0.93 1,299 0.89 33,458 0.93 121 6.56 0.94 1,017. 0.94 1,017 0.94 
English Learner 121 4.60 0.84 107 0.76 2,875 0.85 121 6.17 0.88 101 0.88 101 0.88 
Non-English Learner 121 6.31 0.93 2,305 0.88 51,415 0.93 121 6.48 0.94 1,617 O94 1,617 0.94 
Students with Disabilities 121 5.48 0.92 2,414 0.88 8,084 0.93 121 6.04 0.94 482 0.94 482 0.94 
Students w/o Disabilities 121 6.37 0.93 46,481 0.93 46,158 0.93 121 6.61 0.93 1,237. 0.93 1,237 0.93 
Students Taking 
Accommodated Forms 
A: ASL 121 5.11 0.76 126 0.76 126 0.76 - - - - - - - 
C: Closed Caption 121 6.06 0.92 101 0.92 101 0.92 - - - - - - - 
R: Screen Reader ‘ - ; 7 - 7 - - - . - . - = 
T: Text-to-Speech 121 5.08 0.87 2,313 0.87 2,313 0.87 - - - - - - - 
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Table 8.13 Summary of Test Reliability Estimates for Subgroups: Grade 11 ELA/L 


CBT PBT 
Max. Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 121 6.08 0.93 1,604 0.88 34,137 0.93 121 6.65 0.86 1,069 0.86 1,069 0.86 
Gender 
Male 121 5.79 0.93 1,034 0.87. = 17,737 0.93 121 6.28 0.85 615 0.85 615 0.85 
Female 121 6.36 0.92 570 0.89 16,400 0.93 121 7.12 0.85 454 0.85 454 0.85 
Unknown/Missing = - - = = . - - . - . 7 - - 
Ethnicity 
White 121 6.14 0.93 636 0.90 12,885 0.94 121 6.63 0.85 341 0.85 341 0.85 
African American 121 5.84 0.92 412 0.82 6,661 0.92 121 5.78 0.84 133 0.84 133 0.84 
Asian/Pacific Islander 121 6.58 0.93 2,196 0.93 2,158 0.94 - - - - - - - 
Am. Indian/Alaska Native 121 5.93 0.90 831 0.90 793 0.91 121 7.11 0.84 400 0.84 400 0.84 
Hispanic 121 6.03 0.92 438 0.86 11,051 0.92 121 6.36 0.88 142 0.88 142 0.88 
Multiple 121 6.22 0.93 452 0.92 447 0.94 - - - - - - - 
Special Instructional Needs 
Economically 
Disadvantaged 121 5.93 0.92 724 0.87. 14,287 0.92 121 6.79 0.85 638 0.85 638 0.85 
Not Economically 
Disadvantaged 121 6.18 0.93 880 0.89 19,787 0.94 121 6.49 0.87 414 0.87 414 0.87 
English Learner 121 4.83 0.84 1,709 0.83 1,680 0.85 121 6.72 0.72 130 0.72 130 0.72 
Non-English Learner 121 6.13 0.93 1,555 0.88 32,436 0.93 121 6.66 0.86 924 0.86 924 0.86 
Students with Disabilities 121 5.20 0.91 1,604 0.88 5,541 0.92 121 6.04 0.82 338 0.82 338 0.82 
Students w/o Disabilities 121 6.24 0.93 29,017. 0.92 28,523 0.93 121 6.94 0.84 718 0.84 718 0.84 
Students Taking 
Accommodated Forms 
A: ASL = . ; = - = : = 7 : - 7 
C: Closed Caption - - : - : 7 - : - - 7 7 
R: Screen Reader - - z - - - - - . - - - = 
T: Text-to-Speech 121 4.77 0.88 1,546 0.88 1,546 0.88 - - - - - - - 
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Table 8.14 Summary of Test Reliability Estimates for Subgroups: Grade 3 Mathematics 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max Max. 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample  Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 66 3.53 0.94 2,007. 0.92 34,750 0.94 66 3.75 0.93 5,670 0.90 50,959 0.93 
Gender 
Male 66 3.51 0.94 1,264 0.92 21,748 0.94 66 3.72 0.93 3,414 0.90 25,145 0.94 
Female 66 3.56 0.93 743 0.92 15,413 0.94 66 3.78 0.93 2,256 0.89 25,814 0.93 
Unknown/Missing 
Ethnicity 
White 66 3.63 0.93 30,594 0.92 9,906 0.94 66 3.79 0.92 1,155 0.90 14,263 0.92 
African American 66 3.35 0.93 512 0.89 15,167 0.93 66 3.66 0.92 1,226 0.87 13,145 0.92 
Asian/Pacific Islander 66 3.62 0.94 12,637 0.93 3,446 0.95 66 3.76 0.93 345 0.92 4,665 0.93 
Am. Indian/Alaska Native 66 3.31 0.92 472 0.92 890 0.92 66 3.61 0.91 1,375 0.91 1,375 0.91 
Hispanic 66 3.42 0.93 597 0.89 23,497 0.93 66 3.74 0.92 2,797 0.90 15,994 0.92 
Multiple 66 3.56 0.94 3,049 0.94 971 0.95 66 3.74 0.93 105 0.92 1,491 0.93 
Special Instructional Needs 
Economically 
Disadvantaged 66 3.39 0.93 1,162 0.89 41,931 0.93 66 3.70 0.92 4,613 0.89 32,233 0.92 
Not Economically 
Disadvantaged 66 3.64 0.93 44,795 0.93 13,172 0.94 66 3.79 0.92 1,052 0.91 18,709 0.92 
English Learner 66 3.30 0.92 567 0.90 9,579 0.93 66 3.69 0.92 2,464 0.90 10,447 0.92 
Non-English Learner 66 3.57 0.94 1,437 0.92 24,327 0.94 66 3.76 0.93 3,199 0.89 40,423 0.93 
Students with Disabilities 66 3.23 0.93 1,452 0.90 8,721 0.95 66 3.45 0.91 3,917. 0.89 3,495 0.93 
Students w/o Disabilities 66 3.58 0.94 553 0.93 21,646 0.94 66 3.80 0.93 1,742 0.90 47,339 0.93 
Students Taking 
Accommodated Forms 
A: ASL = - : - = 7 : - : - : - - . 
C: Closed Caption - 7 - - 7 - : - - - : . : 
R: Screen Reader - 7 - 7 - 7 - 7 - 7 - - . - 
T: Text-to-Speech 66 3.35 0.94 34,750 0.94 36,519 0.94 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 66 3.18 0.91 5,137 0.91 5,137 0.91 66 3.41 0.90 886 0.90 886 0.90 
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Table 8.15 Summary of Test Reliability Estimates for Subgroups: Grade 4 Mathematics 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max Max 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample _ Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 66 3.49 0.94 1,966 0.92 102,282 0.94 66 ~=—«3.43 0.93 5,065 0.88 42,538 0.93 
Gender 
Male 66 3.45 0.94 1,259 0.92 51,649 0.94 66 3.39 0.93 3,146 0.88 20,846 0.93 
Female 66 3.53 0.93 50,099 0.93 16,528 0.94 66 3.47 0.93 1,919 0.88 21,692 0.93 
Unknown/Missing - 7 - . - . - = - - - = . = 
Ethnicity 
White 66 3.59 0.93 39,985 0.92 10,215 0.94 66 3.51 0.93 910 0.91 10,688 0.93 
African American 66 3.31 0.93 5,954 0.91 15,531 0.93 66 3.31 0.91 1,435 0.85 10,358 0.91 
Asian/Pacific Islander 66 3.57 0.94 15,964 0.93 3,275 0.95 66 3.52 0.93 216 0.92 3,988 0.93 
Am. Indian/Alaska Native 66 3.29 0.91 539 0.89 923 0.92 66 3.27 0.91 1,238 0.91 1,238 0.91 
Hispanic 66 3.38 0.92 523 0.89 25,229 0.93 66 3.40 0.91 2,367 O.85 15,204 0.92 
Multiple 66 3.53 0.94 928 0.93 3,885 0.94 66 3.54 0.93 1,008 0.93 1,008 0.93 
Special Instructional Needs 
Economically 
Disadvantaged 66 3.35 0.92 1,175 O89 43,485 0.93 66 3.36 0.91 4,189 0.85 28,119 0.92 
Not Economically 
Disadvantaged 66 3.59 0.93 58,061 0.93 13,436 0.94 66 3.54 0.92 874 0.91 14,372 0.93 
English Learner 66 3.06 0.90 6,923 0.88 5,072 0.91 66 3.20 0.89 1,850 0.86 5,671 0.90 
Non-English Learner 66 3.53 0.94 1,530 0.93 26,823 0.94 66 3.46 0.93 3,203 0.89 36,759 0.93 
Students with Disabilities 66 3.14 0.92 14,966 0.90 9,201 0.95 66 3.19 0.90 4,223, 0.87 3,071 0.93 
Students w/o Disabilities 66 3.55 0.93 91,385 0.93 416 0.95 66 3.47 0.93 833 0.89 39,333 0.93 
Students Taking 
Accommodated Forms 
A: ASL = - = - = - = - = - - - - 
C: Closed Caption - - ; - : - - 7 - 7 - - 7 - 
R: Screen Reader = . - e 7 = : 7 : 7 : 7 7 - 
T: Text-to-Speech 66 3.11 0.93 34,358 0.93 35,450 0.94 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 66 2.86 0.90 3,378 0.90 3,378 0.90 66 3.11 0.89 428 0.89 428 0.89 
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Table 8.16 Summary of Test Reliability Estimates for Subgroups: Grade 5 Mathematics 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max Max 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample  Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 66 3.48 0.93 1,784 0.89 46,338 0.93 66 =«3.41 0.92 2,176 0.87 13,450 0.92 
Gender 
Male 66 3.46 0.93 1,112 0.89 33,848 0.94 66 3.39 0.92 1,357 0.87 6,798 0.93 
Female 66 3.50 0.93 672 0.89 21,249 0.93 66 3.42 0.91 819 0.86 6,652 0.92 
Unknown/Missing - e - . - . - = - - - = 7 = 
Ethnicity 
White 66 3.60 0.92 664 0.91 11,337 0.94 66 3.51 0.91 586 0.89 4,687 0.92 
African American 66 3.25 0.91 459 0.78 19,746 0.91 66 3.19 0.89 615 0.84 1,545 0.90 
Asian/Pacific Islander 66 3.61 0.93 13,454 0.93 3,484 0.96 66 3.52 0.93 147 0.93 2,422 0.93 
Am. Indian/Alaska Native 66 3.26 0.90 497 0.89 532 0.91 66 3.23 0.86 1,153 0.86 1,153 0.86 
Hispanic 66 3.35 0.91 501 0.83 19,940 0.91 66 3.33 0.88 745 0.80 3,191 0.90 
Multiple 66 3.53 0.93 3,083 0.93 1,007 0.94 66 3.48 0.93 424 0.93 424 0.93 
Special Instructional Needs 
Economically 
Disadvantaged 66 3.31 0.91 1,106 0.84 31,426 0.91 66 3.27 0.88 1,647 O82 £6,843 0.89 
Not Economically 
Disadvantaged 66 3.61 0.93 677 0.91 14,876 0.94 66 3.55 0.92 529 0.91 6,577 0.92 
English Learner 66 2.98 0.88 391 0.81 5,685 0.89 66 3.15 0.83 540 0.82 1,349 0.84 
Non-English Learner 66 3.51 0.93 1,390 0.90 36,430 0.93 66 3.44 0.92 1,626 0.87 12,011 0.92 
Students with Disabilities 66 3.12 0.90 1414 0.86 12,475 0.93 66 3.17 0.88 1,936 0.86 1,466 0.91 
Students w/o Disabilities 66 3.54 0.93 369 0.92 29,240 0.93 66 3.47 0.92 238 0.89 11,929 0.92 
Students Taking 
Accommodated Forms 
A: ASL = - = - = - = - = - - - - 
C: Closed Caption - - ; - : 7 - 7 - 7 - - 7 7 
R: Screen Reader = = - 5 7 : : 7 : 7 . 7 a. = 
T: Text-to-Speech 66 3.19 0.93 43,585 0.93 44,739 0.93 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 66 2.87 0.86 2,776 0.86 2,776 0.86 - - - - - - - 
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Table 8.17 Summary of Test Reliability Estimates for Subgroups: Grade 6 Mathematics 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max Max 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 66 3.44 0.94 107 0.84 108,745 0.94 66 3.60 0.92 1,603 0.84 15,276 0.93 
Gender 
Male 66 3.39 0.94 639 0.90 54,427 0.94 66 3.54 0.92 1,012 0.83 7,777 0.93 
Female 66 3.49 0.93 393 0.91 17,550 0.94 66 3.66 0.92 591 0.84 7,499 0.93 
Unknown/Missing . 7 7 7 7 - - - - : - 7 : = 
Ethnicity 
White 66 3.62 0.93 34,848 0.92 10,220 0.95 66 3.72 0.92 471 0.85 6,146 0.93 
African American 66 3.10 0.91 244 0.81 15,694 0.92 66 3.22 0.88 460 0.78 1,734 0.90 
Asian/Pacific Islander 66 3.69 0.94 13,456 0.93 2,415 0.96 66 3.85 0.93 113 0.91 2,755 0.93 
Am. Indian/Alaska Native 66 3.09 0.91 295 0.89 1,125 0.91 66 3.27 0.90 1,051 0.90 1,051 0.90 
Hispanic 66 3.23 0.92 290 0.78 27,499 0.92 66 3.43 0.89 502 0.78 3,031 0.91 
Multiple 66 3.51 0.94 2,944 0.93 718 0.95 66 3.70 0.93 542 0.93 542 0.93 
Special Instructional Needs 
Economically 
Disadvantaged 66 3.17 0.91 622 0.82 45,276 0.92 66 3.35 0.89 1,175 0.79 6,840 0.90 
Not Economically 
Disadvantaged 66 3.66 0.93 50,200 0.93 12,755 0.95 66 3.80 0.92 425 0.88 8,421 0.93 
English Learner 66 2.69 0.87 156 0.66 4,428 0.90 66 3.06 0.85 354 0.78 1,116 0.87 
Non-English Learner 66 3.48 0.94 103 0.85 31,502 0.94 66 3.64 0.92 1,240 0.84 14,077 0.93 
Students with Disabilities 66 2.91 0.91 107 0.84 10,656 0.94 66 3.11 0.89 1,466 0.84 1,839 0.92 
Students w/o Disabilities 66 3.53 0.94 205 0.93 23,578 0.94 66 3.71 0.93 132 0.85 13,395 0.93 
Students Taking 
Accommodated Forms 
A: ASL 66 2.68 0.84 107 0.84 107 0.84 - - - - - - - 
C: Closed Caption - - : - - - - 7 7 - 7 - - 
R: Screen Reader . E 7 . 7 = : 7 : = . . . . 
T: Text-to-Speech 66 3.04 0.94 36,286 0.93 37,401 0.94 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 66 2.56 0.84 2,601 0.84 2,601 0.84 - - - - - - - 
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Table 8.18 Summary of Test Reliability Estimates for Subgroups: Grade 7 Mathematics 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 66 3.43 0.92 108 0.70 32,369 0.93 66 «3.41 0.89 1,253 0.79 12,199 0.90 
Gender 
Male 66 3.35 0.93 1,276 0.77 42,143 0.93 66 3.36 0.89 750 0.76 6,160 0.91 
Female 66 3.5 0.92 1,152 0.77. 14,433 0.93 66 3.46 0.89 503 0.81 6,039 0.90 
Unknown/Missing . 7 7 - 7 - - - - 7 - . - = 
Ethnicity 
White 66 3.56 0.92 376 0.89 9,054 0.94 66 3.56 0.90 362 0.83 4,850 0.90 
African American 66 3.14 0.89 7,593 0.88 229 0.93 66 3.04 0.80 363 0.66 1,377 0.83 
Asian/Pacific Islander 66 3.62 0.94 15,176 0.93 2,083 0.96 66 3.61 0.91 2,203 0.91 2,203 0.91 
Am. Indian/Alaska Native 66 3.07 0.88 1,178 0.87 264 0.90 66 3.00 0.81 758 0.81 758 0.81 
Hispanic 66 3.25 0.89 2,300 0.77 12,585 0.91 66 3.22 0.85 391 0.69 2,569 0.87 
Multiple 66 3.47 0.93 3,337 0.92 633 0.94 66 3.53 0.90 427 0.90 427 0.90 
Special Instructional Needs 
Economically 
Disadvantaged 66 3.20 0.89 1,985 0.77 21,174 0.90 66 3.70 0.92 4,613 0.89 32,233 0.92 
Not Economically 
Disadvantaged 66 3.58 0.93 437 0.77. 11,176 0.94 66 3.79 0.92 1,052 0.91 18,709 0.92 
English Learner 66 2.67 0.83 2,428 0.77 3,715 0.85 66 3.69 0.92 2,464 0.90 10,447 0.92 
Non-English Learner 66 3.47 0.92 790 0.90 28,082 0.93 66 3.76 0.93 3,199 0.89 40,423 0.93 
Students with Disabilities 66 2.85 0.89 109 0.65 9,029 0.92 66 3.45 0.91 3,917 0.89 3,495 0.93 
Students w/o Disabilities 66 3.51 0.92 2,307 0.77. 18,641 0.93 66 3.80 0.93 1,742 0.90 47,339 0.93 
Students Taking 
Accommodated Forms 
A: ASL 66 2.55 0.70 108 0.70 108 0.70 - - - - - - - 
C: Closed Caption - - = - = - - 7 - ° - - ’ 7 
R: Screen Reader . . 7 . . . : 7 : = . . . . 
T: Text-to-Speech 66 3.03 0.92 966 0.80 32,369 0.93 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 63 2.57 0.77 2,428 0.77 2,428 0.77 - - - - - - - 
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Table 8.19 Summary of Test Reliability Estimates for Subgroups: Grade 8 Mathematics 


CBT PBT 
Avg. Min. Min. Max. Max Max. Avg. Min. Min. Max Max 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 66 3.26 0.91 884 0.90 83,333 0.92 66 3.51 0.89 1,173 0.75 11,058 0.90 
Gender 
Male 66 3.19 0.92 543 0.88 42,408 0.92 66 3.40 0.89 740 0.76 5,668 0.91 
Female 66 3.32 0.91 12,553 0.90 40,925 0.91 66 3.61 0.88 433 0.73. 5,390 0.89 
Unknown/Missing - . 7 . - 7 - . - 7 - - - - 
Ethnicity 
White 66 3.43 0.91 7,270 0.91 399 0.92 66 3.71 0.90 374 0.80 4,280 0.90 
African American 66 2.96 0.87 187 0.48 12,869 0.87 66 3.11 0.80 302 0.58 1,208 0.85 
Asian/Pacific Islander 66 3.47 0.93 7,815 0.93 1,445 0.94 66 3.70 0.90 1,610 0.90 1,610 0.90 
Am. Indian/Alaska Native 66 2.92 0.86 248 0.82 290 0.89 66 3.24 0.83 763 0.83 763 0.83 
Hispanic 66 3.11 0.88 229 0.76 11,190 0.89 66 3.30 0.85 377 0.71 2,848 0.87 
Multiple 66 3.28 0.92 433 0.91 491 0.93 66 3.65 0.92 334 0.92 334 0.92 
Special Instructional Needs 
Economically 
Disadvantaged 66 3.06 0.88 474 0.73 37,774 0.89 66 3.25 0.84 845 0.72 5,675 0.86 
Not Economically 
Disadvantaged 66 3.43 0.92 8,744 0.91 403 0.92 66 3.75 0.90 326 0.78 5,371 0.90 
English Learner 66 2.66 0.82 111 0.75 4,140 0.84 66 2.91 0.73 260 0.65 1,063 0.75 
Non-English Learner 66 3.30 0.91 24,384 0.90 77,217 0.92 66 3.56 0.89 909 0.77. 9,955 0.90 
Students with Disabilities 66 2.79 0.86 12,020 0.81 10,573 0.90 66 2.88 0.83 1,088 0.75 1,494 0.89 
Students w/o Disabilities 66 3.35 0.91 16,253 0.90 175 0.93 66 3.66 0.90 9538 0.90 9,538 0.90 
Students Taking 
Accommodated Forms 
A: ASL 66 2.45 0.59 105 0.59 105 0.59 - - - - - - - 
C: Closed Caption - . - 7 ‘ - - 7 - - - - - - 
R: Screen Reader - r - - - 7 : : - 7 - 7 7 = 
T: Text-to-Speech 66 3.10 0.90 28,298 0.90 28,826 0.90 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 66 2.48 0.75 2,464 0.75 2,464 0.75 - - - - - - - 
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CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 81 3.61 0.94 488 0.85 67,566 0.94 81 3.73 0.92 406 0.86 5,623 0.92 
Gender 
Male 81 3.59 0.94 287 0.87 34,399 0.95 81 3.70 0.92 248 0.84 2,953 0.93 
Female 81 3.63 0.93 201 0.82 33,167 0.94 81 3.76 0.91 158 0.88 2,670 0.91 
Unknown/Missing : 7 7 7 7 - - - - 7 - : : = 
Ethnicity 
White 81 3.87 0.93 124 0.88 24,719 0.94 81 3.98 0.92 1,446 0.92 1,446 0.92 
African American 81 3.16 0.90 123 0.88 10,781 0.91 81 3.30 0.86 642 0.86 642 0.86 
Asian/Pacific Islander 81 3.94 0.95 11,302 0.94 889 0.96 81 4.10 0.91 1,407 0.91 1,407 0.91 
Am. Indian/Alaska Native 81 2.87 0.84 174 0.79 916 0.86 81 93.15 0.80 168 0.80 168 0.80 
Hispanic 81 3.25 0.90 166 0.77. =17,703 0.91 81 3.38 0.87 290 0.79 1,748 0.88 
Multiple 81 3.72 0.94 197 0.93 181 0.94 81 4.00 0.92 179 0.92 179 0.92 
Special Instructional Needs 
Economically 
Disadvantaged 81 3.20 0.90 210 0.87 25,298 0.91 81 3.40 0.86 261 0.77 2,423 0.87 
Not Economically 
Disadvantaged 81 3.83 0.94 202 0.86 42,107 0.94 81 3.96 0.92 145 0.90 3,180 0.92 
English Learner 81 2.76 0.82 3,920 0.75 3,469 0.88 81 83.11 0.75 273 0.73 615 0.76 
Non-English Learner 81 3.65 0.94 419 0.86 62,786 0.94 81 3.82 0.92 132 0.92 4,952 0.92 
Students with Disabilities 81 3.07 0.90 4,462 0.85 9,085 0.93 81 3.32 0.90 794 0.90 145 0.92 
Students w/o Disabilities 81 3.69 0.94 102 0.73 58,262 0.94 81 3.80 0.91 261 0.72 4,808 0.92 
Students Taking 
Accommodated Forms 
A: ASL 81 2.97 0.78 151 0.78 151 0.78 - - - - - - - 
C: Closed Caption - - ‘ - 7 7 7 7 7 - 7 - - 
R: Screen Reader = . 7 7 e . . . : - : 7 e - 
T: Text-to-Speech 80 3.84 0.92 11,069 0.92 11,757 0.93 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 80 2.92 0.69 2,618 0.69 2,618 0.69 81 3.09 0.72 255 0.72 255 0.72 
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CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Max.Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample  Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 80 3.48 0.94 204 0.86 35,202 0.95 81 3.6 0.93 1,306 0.93 1,306 0.93 
Gender 
Male 80 3.45 0.94 102 0.88 17,671 0.95 81 3.57 0.94 675 0.94 675 0.94 
Female 80 3.51 0.93 102 0.82 17,531 0.94 81 3.63 0.93 631 0.93 631 0.93 
Unknown/Missing 7 e - . - z - . - - - . 7 7 
Ethnicity 
White 80 3.63 0.93 15,806 0.93 15,030 0.94 81 3.33 0.90 241 0.90 241 0.90 
African American 80 3.01 0.90 104 0.81 3,850 0.92 81 2.71 0.83 102 0.83 102 0.83 
Asian/Pacific Islander 80 4.03 0.93 5,614 0.93 302 0.96 81 4.14 0.90 428 0.90 428 0.90 
Am. Indian/Alaska Native 80 2.93 0.87 108 0.79 707 0.89 81 3.17 0.85 168 0.85 168 0.85 
Hispanic 80 3.08 0.90 3,591 0.84 9,564 0.91 81 3.34 0.92 317 0.92 317 0.92 
Multiple 80 3.76 0.94 746 0.94 677 0.95 - - - - - - - 
Special Instructional Needs 
Economically 
Disadvantaged 80 3.04 0.90 158 0.84 11,443 0.91 81 3.27 0.90 566 0.90 566 0.90 
Not Economically 
Disadvantaged 80 3.68 0.94 24,842 0.94 23,632 0.94 81 3.84 0.93 718 0.93 718 0.93 
English Learner 81 2.58 0.82 2,125 0.74 1,131 0.88 - - - - - - - 
Non-English Learner 80 3.52 0.94 180 0.85 33,524 0.95 81 3.62 0.93 1,191 0.93 1,191 0.93 
Students with Disabilities 80 2.84 0.91 2,005 0.81 4,677 0.93 81 2.98 0.90 262 0.90 262 0.90 
Students w/o Disabilities 80 3.57 0.94 122 0.82 30,378 0.94 81 3.74 0.93 1,025 0.93 1,025 0.93 
Students Taking 
Accommodated Forms 
A: ASL = - = - = - = - = - - - - 
C: Closed Caption - - ; - : - - 7 - 7 - - 7 7 
R: Screen Reader 7 = 7 F 7 . : 7 : 7 . 7 a. = 
T: Text-to-Speech 81 3.09 0.93 5,910 0.93 5,511 0.94 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 81 2.52 0.68 1,677 0.68 1,677 0.68 - - - - - - - 
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CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min Min. Max. Max. 
Max.Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample _ Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 81 3.51 0.93 4,818 0.92 35,156 0.93 81 4.01 0.93 2,051 0.93 2,051 0.93 
Gender 
Male 81 3.49 0.94 2,564 0.93 17,538 0.94 81 3.93 0.94 1,115 0.94 1,115 0.94 
Female 81 3.52 0.92 2,254 0.91 17,379 0.92 81 4.09 0.90 936 0.90 936 0.90 
Unknown/Missing = 7 - = - . - . - - - = 7 . 
Ethnicity 
White 81 3.67 0.92 1497 0.92 14,722 0.93 81 4.16 0.92 1,191 0.92 1,191 0.92 
African American 81 3.07 0.87 5,937 0.86 807 0.88 81 3.52 0.88 246 0.88 246 0.88 
Asian/Pacific Islander 81 3.97 0.94 227 0.94 3,771 0.94 81 4.34 0.93 134 0.93 134 0.93 
Am. Indian/Alaska Native 81 3.06 0.81 684 0.80 680 0.84 81 3.52 0.87 209 0.87 209 0.87 
Hispanic 81 3.23 0.87 2,026 0.87 1,200 0.88 81 3.67 0.92 186 0.92 186 0.92 
Multiple 81 3.66 0.93 650 0.93 665 0.94 - - - - - - - 
Special Instructional Needs 
Economically 
Disadvantaged 81 3.18 0.87 12,065 0.86 2,485 0.88 81 3.63 0.89 595 0.89 595 0.89 
Not Economically 
Disadvantaged 81 3.65 0.93 2,332 0.93 23,291 0.94 81 4.15 0.93 1,445 0.93 1,445 0.93 
English Learner 81 2.78 0.85 238 0.76 902 0.87 - - - - - - - 
Non-English Learner 81 3.53 0.93 3,760 0.92 34,082 0.93 81 4.04 0.93 1,978 0.93 1,978 0.93 
Students with Disabilities 81 2.98 0.89 1,090 0.81 3,948 0.90 81 3.36 0.92 323 0.92 323 0.92 
Students w/o Disabilities 81 3.57 0.93 3,723 0.92 31,096 0.93 81 4.11 0.92 1,720 0.92 1,720 0.92 
Students Taking 
Accommodated Forms 
A: ASL = - = - = - = - = - - - - 
C: Closed Caption - - ; - : - - 7 - 7 - - 7 7 
R: Screen Reader 7 = - 7 e . : 7 : 7 . 7 a. = 
T: Text-to-Speech 81 3.44 0.92 4,224 0.92 3,749 0.92 - - - - - - - 
Students Taking Translated 
Forms 
Spanish-Language Form 81 2.73 0.83 814 0.83 814 0.83 - - - - - - - 
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Table 8.23 Summary of Test Reliability Estimates for Subgroups: Integrated Mathematics | 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 81 3.53 0.92 425 0.71 9,376 0.93 S - - - - - - 
Gender 
Male 81 3.52 0.92 268 0.74 4,995 0.93 - - - - - - - 
Female 81 3.54 0.92 157 0.67 4,381 0.93 - - - - - - - 
Unknown/Missing = . - z - . - 7 - - - = . - 
Ethnicity 
White 81 3.53 0.93 224 0.93 224 0.93 - - - - - - - 
African American 81 3.34 0.91 692 0.91 692 0.91 - - - - - - - 
Asian/Pacific Islander 81 3.81 0.93 3,735 0.93 3,735 0.93 - - - - - - - 
Am. Indian/Alaska Native 81 3.26 0.88 103 0.88 103 0.88 - - - - - - - 
Hispanic 81 3.28 0.89 260 0.63 4,375 0.90 - - - - - - - 
Multiple 81 3.64 0.93 243 0.93 243 0.93 - - - - - - - 
Special Instructional Needs 
Economically 
Disadvantaged 81 3.31 0.89 317 0.65 4,772 0.91 - - - - - - - 
Not-economically 
Disadvantaged 81 3.73 0.93 108 0.80 4,604 0.93 - - - - - - - 
English Learner 81 2.88 0.77 234 0.70 1,707 0.79 - - - - - - - 
Non-English Learner 81 3.62 0.92 154 0.73 5,601 0.92 - - - - - - - 
Students with Disabilities 81 2.97 0.85 284 0.66 935 0.91 - - - - - - - 
Students w/o Disabilities 81 3.59 0.93 141 0.78 8,441 0.93 - - - - - - - 
Students Taking 
Accommodated Forms 
A: ASL . - ‘ - : - : 7 7 - - - 7 - 
C: Closed Caption - 7 : a. : : - : - - = 
R: Screen Reader 7 = - . - . 7 : 7 = 7 z = 
81 2.88 0.68 368 0.68 368 0.68 - - - - - - - 


T: Text-to-Speech 
Students Taking Translated 
Forms 


Spanish-Language Form 
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Table 8.24 Summary of Test Reliability Estimates for Subgroups: Integrated Mathematics II 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 80 3.73 0.90 1,914 0.90 1,914 0.90 go. 3.21 0.86 182 0.86 182 0.86 
Gender 
Male 81 3.49 0.94 2,564 0.93 17,538 0.94 - - - - - - - 
Female 81 3.52 0.92 2,254 0.91 17,379 0.92 - - - - - - - 
Unknown/Missing = 7 = . - - 7 7 7 - : : = - 
Ethnicity 
White - : ‘ 7 : 7 : - = 
African American 80 3.47 0.89 178 0.89 178 0.89 = = = = = Zz = 
Asian/Pacific Islander 80 3.89 0.87 995 0.87 995 0.87 = = - - 7 - 7 
Am. Indian/Alaska Native 7 7 7 7 - = = 80 3.21 0.86 178 0.86 178 0.86 
Hispanic 80 3.49 0.89 564 0.89 564 0.89 = = = = - = 7 
Multiple . 7 : 7 : : 2 . - - : 
Special Instructional Needs 
Economically 
Disadvantaged 80 3.55 0.89 758 0.89 758 0.89 - - - - - - - 
Not Economically 
Disadvantaged 80 3.83 0.89 1,156 0.89 1,156 0.89 - = - = - - - 
English Learner : - - ; : 7 7 ; : - : 
Non-English Learner 80 3.74 0.89 1,817 0.89 1,817 0.89 80 3.51 0.88 101 0.88 101 0.88 
Students with Disabilities - - - - - : : : : - - 


Students Taking 
Accommodated Forms 


A: ASL 

C: Closed Caption 

R: Screen Reader 

T: Text-to-Speech 
Students Taking Translated 
Forms 


Spanish-Language Form 
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Table 8.25 Summary of Test Reliability Estimates for Subgroups: Integrated Mathematics III 


CBT PBT 
Avg. Min. Min. Max. Max. Max. Avg. Min. Min. Max. Max. 
Max. Raw Avg. Relia- Sample Relia- Sample Relia- Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility Score SEM bility Size bility Size bility 
Total Group 81 3.89 0.93 477 0.93 477 0.93 é - - - F E : 
Gender 
Male 81 3.92 0.93 250 0.93 250 0.93 - : - - “ Z : 
Female 81 3.86 0.94 227 0.94 227 0.94 - - . - : : 2 
Unknown/Missing - - - - - - 2 a Z - - . : _ 
Ethnicity 
White - 7 - - - 7 - 2 - 2 S . = _ 
African American : - - - - E - E 7 : : - E . 
Asian/Pacific Islander 81 4.60 0.87 163 0.87 163 0.87 - = - : E : - 
Am. Indian/Alaska Native 81 3.18 0.78 100 0.78 100 0.78 - - : z 2 : - 
Hispanic 81 3.15 0.90 144 0.90 144 0.90 - 7 : = 2 2 - 
Multiple = - = - - - - ‘ 2 = z . - _ 
Special Instructional Needs 
Economically 
Disadvantaged 81 3.39 0.85 296 0.85 296 0.85 - - - - 2 S 2 
Not Economically 
Disadvantaged 81 4.55 0.89 181 0.89 181 0.89 - - : 2 2 : - 
English Learner - : - - - ss - : : 2 Z - : : 
Non-English Learner 81 3.96 0.93 450 0.93 450 0.93 - 2 - : 2 - 
Students with Disabilities : : - - - “ 2 - J : 7 - 
Students w/o Disabilities 81 3.95 0.93 448 0.93 448 0.93 - - - zs - - 


Students Taking 
Accommodated Forms 


A: ASL 

C: Closed Caption 

R: Screen Reader 

T: Text-to-Speech 
Students Taking Translated 
Forms 


Spanish-Language Form 
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8.5 Reliability Results for English Language Arts/Literacy Subscores 


PARCC developed subclaims in addition to major claims based on the Common Core State Standards. 
English language arts/literacy (ELA/L) has two major claims relating to Reading Complex Text and 
Writing. The major claim for Reading Complex Text is that students read and comprehend a range of 
sufficiently complex texts independently. The major claim for Writing is that students write effectively 
when using and/or analyzing sources. Refer to Table 8.26 for a summary of the ELA/L claims and 
subclaims. 


Table 8.26 Descriptions of ELA/L Claims and Subclaims 


English Language Arts/Literacy 


Major Claim Subclaim Description 


Reading Reading Literature Students demonstrate comprehension and draw evidence from 
readings of grade-level, complex literary text. 


Reading Reading Information Students demonstrate comprehension and draw evidence from 
readings of grade-level, complex informational text. 
Reading Reading Vocabulary Students use context to determine the meaning of words and 
phrases. 
Writing Writing Written Students produce clear and coherent writing in which the 
Expression development, organization, and style are appropriate to the task, 
purpose, and audience. 
Writing Writing Knowledge Students demonstrate knowledge of conventions and other 
Language and important elements of language. 


Conventions 


Reliability indices were calculated for each major claim and subclaim. Table 8.27 presents the average 
reliability estimates for all forms of the test at the specified grade and testing mode for the ELA/L tests. 
In order to assist in understanding the reliability estimates, the average maximum number of points for 
each major claim and subclaim are also provided. 


The reliabilities for the Reading Complex Text claim for grades 3 through 11 range from .87 to .91 for 
CBT and from .80 to .93 for PBT. The median reliability across all grades and modes is .90. The reliability 
for grades 3 through 5 range from .85 to .89, and the average reliability for grades 6 through 11 ranges 
from .85 to .93. Grade 11 reliability for the PBT is .80. 


The Writing claim reliabilities are generally lower than those for the Reading claim. The median 
reliability across all grades and modes is .85. The reliabilities for the Writing claim for grades 3 and 4 are 
based on 27 points and 33 points, respectively, and the average reliabilities for the grades 5 through 11 
Writing claims are based on 36 points. The reliabilities for grades 3 and 5 range from .79 to .83, and the 
average reliability for grades 6 through 11 ranges from .81 to .89. Taking the number of points into 
consideration, the per-point information of the two claims are quite similar, as are the per-point 
information when comparing grades 3 through 5 with grades 6 through11. 


March 27, 2018 Page 124 


PARCC 2017 Technical Report 


The reliabilities of the Reading Literature subclaim scores over testing modes and grades have a median 
of .76, and the reliabilities vary from .61 to .79. For seven of the nine PBTs, the Reading Literature 
subclaim was based on less than 25 points. The reliabilities for these PBTs ranged from .61 to .72. The 
reliabilities for tests with 25 or more points ranged from .71 to .79. For grades 3 through 5, the Reading 
Information subclaim reliabilities are based on 19 to 26 points and the reliability is typically lower than 
for grades 6 through 11, where the Reading Information subclaim is based on 30 to 40 points. The 
median average reliability for Reading Information is .79 overall. Once again, when taking the number of 
points into consideration, the per-point information of the claim is quite similar when comparing grades 
3 through 5 with grades 6 through 11. The Reading Vocabulary subclaim is based on the fewest points, 
ranging from 12 to 24 points. The subclaim average reliabilities have a median of .67 and range from .48 
to .78. The lower reliabilities for the Reading Vocabulary subclaim are reflected in the smaller range of 
raw score points. 


The Writing Written Expression subclaim is based on 27 and 33 points for grades 3 and 4—5 CBT 
respectively, and 36 points for grade 4 and 5 PBT and grades 6 through 11. The median of the average 
reliabilities for the tests with less than 36 points is .77, and the median of the average reliabilities for 
grades 6 through 11 is .85. The Writing Knowledge of Language and Conventions subclaims are all based 
on nine points. The average reliabilities are consistent, varying from .78 to .90, with a median of .84. 
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Table 8.27 Average ELA/L Reliability Estimates for Total Test and Subscores 


Writing: 
Reading: Reading: Reading: Reading: Writing: Writing: Knowledge 
Total Literature Information Vocabulary Total Written Expression Language and 


Conventions 


Average Average Average Average Average Average Average 
Grade __ Testing Max Average Max Average Max Average Max Average Max Average Max Average Max Average 
Level Mode Possible Reliability Possible —_— Reliability Possible — Reliability | Possible — Reliability Possible — Reliability © Possible Reliability Possible _Reliability 
Raw Raw Raw Raw Raw Raw Raw 
Score Score Score Score Score Score Score 
CBT 58 0.90 25 0.79 20 0.75 13 0.73 36 0.81 27 0.74 9 0.83 
: PBT 58 0.89 27 0.77 19 0.71 12 0.74 36 0.80 27 0.72 9 0.80 
CBT 64 0.89 26 0.76 24 0.77 14 0.63 42 0.83 33 0.80 9 0.84 
: PBT 64 0.86 24 0.66 24 0.72 16 0.67 45 0.79 36 0.73 9 0.78 
CBT 63 0.87 25 0.71 24 0.74 14 0.62 42 0.85 33 0.79 9 0.86 
: PBT 64 0.85 24 0.62 26 0.73 14 0.59 45 0.81 36 0.72 9 0.80 
CBT 76 0.91 26 0.78 34 0.82 17 0.70 45 0.87 36 0.86 9 0.88 
6 PBT 76 0.90 22 0.73 34 0.81 20 0.71 45 0.84 36 0.81 9 0.83 
CBT 76 0.90 25 0.75 35 0.81 17 0.64 45 0.87 36 0.86 9 0.89 
. PBT 76 0.88 24 0.72 32 0.75 20 0.68 45 0.84 36 0.81 9 0.82 
CBT 76 0.90 29 0.79 32 0.78 15 0.62 45 0.88 36 0.88 9 0.89 
: PBT 76 0.89 16 0.72 40 0.80 20 0.69 45 0.85 36 0.82 9 0.83 
CBT 76 0.90 27 0.78 36 0.82 13 0.61 45 0.89 36 0.89 9 0.90 
7 PBT 76 0.91 28 0.78 34 0.80 14 0.69 45 0.86 36 0.83 9 0.84 
CBT 76 0.90 26 0.77 30 0.81 20 0.67 45 0.88 36 0.88 9 0.88 
a PBT 76 0.93 22 0.78 30 0.86 24 0.78 45 0.87 36 0.87 9 0.88 
CBT 76 0.90 24 0.75 34 0.81 18 0.67 45 0.87 36 0.86 9 0.86 
in PBT 76 0.80 20 0.61 36 0.63 20 0.48 45 0.83 36 0.79 9 0.79 
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8.6 Reliability Results for Mathematics Subscores 


For mathematics, there are four subclaims related to the major claim that students are on track or ready 
for college and careers: 


- Subclaim A: Students solve problems involving the major content for their grade level with 
connections to the Standards for Mathematical Practice. 


- Subclaim B: Students solve problems involving the additional and supporting content for their grade 
level with connections to the Standards for Mathematical Practice. 


- Subclaim C: Students express grade/course-level appropriate mathematical reasoning by 
constructing viable mathematical arguments and critiquing the reasoning of others, and/or 
attending to precision when making mathematical statements. 


-  Subclaim D: Students solve real-world problems with a degree of difficulty appropriate to the 
grade/course by applying knowledge and skills articulated in the standards and by engaging 
particularly in the modeling practice. 


Reliability estimates were calculated for each subclaim for mathematics. Table 8.28 presents the 
average reliability estimates for mathematics subclaims by mode (CBT and PBT) and grade/subject. 
The sample sizes for Integrated Mathematics | and III PBT were not sufficient for reliability analyses. 


Subclaims with greater numbers of points tend to have greater reliability estimates. The Major 
Content subclaim has the largest number of points for each assessment and, accordingly, has higher 
average reliabilities than the other three subclaims. For all grades and modes, the median of the 
average reliabilities for the subclaim is .83. The Major Content reliabilities were higher for the grade 
3 —8 tests and lower for the six end-of-course (EOC) assessments. The medians of the average 
reliabilities for the Major Content subclaim for the grade level test versus the EOC tests are .86 and 
.81 respectively. 


The median of the average reliabilities for the Additional and Supporting Content subclaim for 
grades 3 through 8 is .65, and the median of the average reliabilities for the traditional EOC tests is 
.73. Due to the similarity in the number of items representing the Mathematics Reasoning subclaim 
across grades and courses, the reliabilities for Mathematics Reasoning are less variable than those 
for the Additional and Supporting Content subclaim, and range from .48 for Integrated Mathematics 
Il PBT test forms to .75 for the grade 6 PBT test forms. The median of the average reliabilities for this 
subclaim is .71 


For the Modeling Practice subclaim, the median of the average reliabilities is .62 for grades 3 
through 8, and the medians of the averages are .70 and .66 for the traditional and integrated 
mathematics EOC test forms, respectively. 
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Table 8.28 Average Mathematics Reliability Estimates for Total Test and Subscores 


Additional & Supporting 


Major Content ankont Mathematics Reasoning Modeling Practice 
Gisae AVErAaRE Max Avetaae PuerARE Max Average PNETARE Max Average VETERE Max Average 
iaiel Testing Mode Possible Raw Reliability Possible Raw Reliability Possible Raw Reliability Possible Raw Reliability 
Score Score Score Score 
CBT 29 0.89 11 0.75 14 0.71 12 0.68 
: PBT 28 0.87 12 0.75 14 0.67 12 0.66 
CBT 31 0.89 9 0.69 14 0.73 12 0.63 
: PBT 31 0.87 9 0.64 14 0.72 12 0.60 
CBT 30 0.87 10 0.63 14 0.72 12 0.65 
PBT 30 0.84 10 0.62 14 0.66 12 0.53 
CBT 26 0.87 14 0.72 14 0.73 12 0.70 
: PBT 26 0.83 14 0.68 14 0.75 12 0.64 
CBT 29 0.85 11 0.65 14 0.74 12 0.54 
: PBT 29 0.76 11 0.61 14 0.72 12 0.45 
CBT 27 0.81 13 0.63 14 0.70 12 0.60 
‘ PBT 27 0.77 13 0.58 14 0.62 12 0.58 
CBT 26 0.82 15 0.72 14 0.74 18 0.75 
i PBT 26 0.78 15 0.63 14 0.65 18 0.70 
CBT 30 0.86 18 0.76 14 0.74 18 0.68 
_ PBT 30 0.87 19 0.71 14 0.66 18 0.71 
CBT 25 0.82 20 0.77 14 0.70 18 0.65 
= PBT 23 0.80 20 0.74 14 0.68 18 0.66 
ana CBT 27 0.77 20 0.77 14 0.69 18 0.73 
PBT i. . ° 5 ; : . . 
‘a CBT 31 0.78 17 0.65 14 0.57 18 0.62 
PBT 29 0.62 19 0.63 14 0.48 18 0.59 
_— CBT 26 0.83 23 0.73 14 0.74 18 0.69 
PBT : : : : : : : : 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated Mathematics II, M3 = Integrated Mathematics Ill. 
Integrated Mathematics | and III PBT had insufficient sample sizes. 
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8.7 Reliability of Classification 


The reliability of the classifications for the test takers was calculated using the computer program BB- 
CLASS (Brennan, 2004), which operationalizes a statistical method developed by Livingston and Lewis 
(1993, 1995). As Livingston and Lewis (1993, 1995) explain, this method uses information from the 
administration of one test form (i.e., distribution of scores, the minimum and maximum possible scores, 
the cut points used for classification, and the reliability coefficient) to estimate two kinds of statistics, 
decision accuracy and decision consistency. Decision accuracy refers to the extent to which the 
classifications of test takers based on their scores on the test form agree with the classifications made 
on the basis of the classifications that would be made if the test scores were perfectly reliable. Decision 
consistency refers to the agreement between these classifications based on two non-overlapping, 
equally difficult forms of the test. 


Decision consistency values are always lower than the corresponding decision accuracy values, because 
in decision consistency, both of the classifications of the student are based on scores that depend on 
which form of the test the student took. In decision accuracy, only one of the classifications is based on 
a score that can vary in this way. It is not possible to know which students were accurately classified, but 
it is possible to estimate the proportion of the students who were accurately classified. Similarly, it is not 
possible to know which students would be consistently classified if they were retested with another 
form, but it is possible to estimate the proportion of the students who would be consistently classified. 


English Language Arts/Literacy 

Table 8.29 provides information about the accuracy and the consistency of two classifications made on 
the basis of the scores on the grades 3 through 11 ELA/L assessments. The columns labeled “Exact level” 
provide the classification of the student into one of five achievement levels. The columns labeled “Level 
4 or higher vs. 3 or lower” provide the classification of the student as being either in one of the upper 
two levels (Levels 4 and 5) or in one of the lower three levels (Levels 1, 2, and 3). 


The table shows that for classifying each student into one of the five achievement levels, the proportion 
accurately classified ranges from .67 to .78; the proportion who would be consistently classified on two 
different test forms ranges from .56 to .69. For classifying each student simply as being at Level 4 or 
higher vs. being at Level 3 or lower, the proportion accurately classified ranges from .89 to .93; the 
proportion who would be consistently classified this way on two different test forms ranges from .85 to 
90. 
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Table 8.29 Reliability of Classification: Summary for ELA/L 


Decision Accuracy: Decision Consistency: 
Proportion Accurately Proportion Consistently 
Classified Classified 
Level 4 or Level 4 or 
Grade Testing higher vs. 3 or higher vs. 3 or 

Level Mode Exact level lower Exact level lower 
3 CBT 0.75 0.92 0.66 0.88 
PBT 0.72 0.91 0.63 0.87 

i CBT 0.73 0.91 0.63 0.87 
PBT 0.70 0.90 0.59 0.85 

: CBT 0.75 0.91 0.66 0.87 
PBT 0.72 0.90 0.63 0.85 

6 CBT 0.78 0.92 0.69 0.89 
PBT 0.76 0.91 0.67 0.88 

j CBT 0.75 0.92 0.65 0.89 
PBT 0.71 0.91 0.61 0.87 

3 CBT 0.75 0.92 0.66 0.89 
PBT 0.72 0.91 0.62 0.88 

9 CBT 0.77 0.93 0.68 0.90 
PBT 0.75 0.93 0.66 0.90 

‘0 CBT 0.73 0.92 0.64 0.89 
PBT 0.75 0.93 0.66 0.90 

11 CBT 0.75 0.92 0.66 0.89 
PBT 0.67 0.89 0.56 0.85 


Table 8.30 provides more detailed information about the accuracy and the consistency of the 
classification of students into proficiency levels for ELA/L grade 3. Each cell in the 5-by-5 table shows the 
estimated proportion of students who would be classified into a particular combination of proficiency 
levels. The sum of the five bold values on the diagonal should equal the level of decision accuracy or 
consistency presented in Table 8.29. For “Level 4 and higher vs. 3 and lower” found in Table 8.27, the 
sum of the shaded values in Table 8.28 should equal the level of decision accuracy or consistency 
presented in Table 8.29. Note that the sums based on values in Table 8.28 may not match exactly to the 
values in Table 8.29 due to truncation and rounding. 


Detailed information for all ELA/L spring results are provided in Appendix 8 Tables A.8.1 to A.8.9. Fall 
block results for ELA/L grades 9 through 11 are provided in the addendum to Section 8. The structure of 
these tables is the same as that of Table 8.28 and the values in the tables should be interpreted in the 
same manner as Table 8.30. 
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an : Category 

Summative Level1 Level2 Level 3 Level 4 Level 5 Total 
Scale Score 

650 — 699 0.15 0.02 0.00 0.00 0.00 0.18 
Decision 700 — 724 0.03 0.11 0.04 0.00 0.00 0.19 
Accuracy 725 — 749 0.00 0.04 0.15 0.04 0.00 0.23 
750 — 809 0.00 0.00 0.04 0.31 0.02 0.37 
ear 810 — 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 — 699 0.15 0.04 0.00 0.00 0.00 0.19 
Decision 700 — 724 0.04 0.09 0.05 0.01 0.00 0.19 
Consistency 725 — 749 0.00 0.04 0.11 0.05 0.00 0.22 
750 — 809 0.00 0.01 0.06 0.28 0.02 0.37 
810 — 850 0.00 0.00 0.00 0.02 0.02 0.04 
650 — 699 0.17 0.03 0.00 0.00 0.00 0.19 
Decision 700 — 724 0.04 0.11 0.04 0.00 0.00 0.20 
Accuracy 725 — 749 0.00 0.04 0.12 0.05 0.00 0.22 
750 — 809 0.00 0.00 0.04 0.29 0.02 0.36 
a 810 — 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 — 699 0.16 0.04 0.01 0.00 0.00 0.21 
Decision 700 — 724 0.04 0.09 0.05 0.01 0.00 0.19 
Consistency 725-749 0.01 0.05 0.09 0.06 0.00 0.21 
750 — 809 0.00 0.01 0.06 0.26 0.02 0.35 
810 — 850 0.00 0.00 0.00 0.02 0.02 0.04 


Note: This table includes the same information as Table A.8.1. The sum of the five bold values on the 
diagonal should equal the level of decision accuracy or consistency presented in Table 8.27. For “Level 4 
and higher vs. 3 and lower” presented in Table 8.27, the sum of the shaded values in Table 8.28 should 
equal the level of decision accuracy or consistency presented in Table 8.27. Any differences between the 
sums based on values in Table 8.28 and the values in Table 8.27 are due to truncation and rounding. 


Mathematics 

Table 8.31 provides information about the accuracy and the consistency of two classifications made on 
the basis of the scores on the mathematics assessments. For the grades 3 through 8 mathematics tests, 
the table shows that for classifying each student into one of the five achievement levels, the proportion 
accurately classified ranges from .69 to .79; the proportion who would be consistently classified on two 
different test forms ranges from .59 to .71. For the six high school mathematics courses, the table shows 
that for classifying each student into one of the five achievement levels, the proportion accurately 
classified ranges from .71 to .81; the proportion who would be consistently classified on two different 
test forms ranges from .61 to .73. 
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For classifying each student simply as being at Level 4 or higher vs. being at Level 3 or lower, for the 
grades 3 through 8 mathematics tests, the proportion accurately classified ranges from .91 to .93; the 
proportion who would be consistently classified on two different test forms ranges from .88 to .91. For 
the six high school mathematics courses, the proportion accurately classified as being at Level 4 or 
higher vs. being at Level 3 or lower ranges from .90 to .94; the proportion who would be consistently 
classified on two different test forms ranges from .86 to .92. 


Appendix 8 tables A.8.10 to A.8.21 provide more detailed information about the accuracy and the 
consistency of the classification of students into proficiency levels for mathematics. Each cell in the 5-by- 
5 table shows the estimated proportion of students who would be classified into a particular 
combination of proficiency levels. Fall block results for Algebra |, Geometry, and Algebra II are provided 
in the addendum to Section 8. 
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Table 8.31 Reliability of Classification: Summary for Mathematics 


Decision Accuracy: Decision Consistency: 
Proportion Accurately Proportion Consistently 
Classified Classified 
Level 4 or Level 4 or 
Grade Testing higher vs. 3 or higher vs. 3 or 
Level Mode Exact Level lower Exact Level lower 
3 CBT 0.77 0.93 0.68 0.90 
PBT 0.76 0.92 0.66 0.89 
i CBT 0.79 0.93 0.71 0.90 
PBT 0.77 0.93 0.68 0.91 
: CBT 0.77 0.92 0.68 0.89 
PBT 0.76 0.93 0.66 0.91 
6 CBT 0.79 0.93 0.70 0.90 
PBT 0.76 0.92 0.66 0.89 
j CBT 0.77 0.92 0.68 0.89 
PBT 0.74 0.92 0.64 0.88 
3 CBT 0.74 0.92 0.65 0.89 
PBT 0.69 0.91 0.59 0.88 
AL CBT 0.79 0.93 0.71 0.90 
PBT 0.76 0.92 0.67 0.89 
60 CBT 0.80 0.93 0.72 0.90 
PBT 0.81 0.93 0.73 0.90 
AD CBT 0.77 0.93 0.69 0.91 
PBT 0.76 0.93 0.67 0.89 
at CBT 0.75 0.93 0.66 0.90 
PBT : : : : 
Ke CBT 0.74 0.90 0.64 0.86 
PBT 0.71 0.94 0.61 0.92 
M3 CBT 0.78 0.93 0.70 0.90 
PBT 3 - - : 
Note: A1 = Algebra |, GO = Geometry, A2 = Algebra Il, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics II, M3 = Integrated Mathematics Ill. “--” means insufficient sample size (< 100 students). 
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8.8 Inter-rater Agreement 


Inter-rater agreement is the agreement between the first and second scores assigned to student 
responses. Inter-rater agreement measurements include exact, adjacent, and nonadjacent agreement. 
Pearson scoring staff used these statistics as one factor in determining the needs for continuing training 
and intervention on both individual and group levels. Table 8.32 displays both PARCC’s expectations and 
the actual spring 2017 agreement percentages for perfect agreement and perfect plus adjacent 


agreement. 
Table 8.32 Inter-rater Agreement Expectations and Results 
ies Within 
; Within One 
. Score Point Perfect Agreement Perfect ; One 
Subject . Point : 
Range Expectation Agreement Result . Point 
Expectation 
Result 
Mathematics 0-1 90% 97% 96% 100% 
Mathematics 0-2 80% 94% 96% 100% 
Mathematics 0-3 70% 93% 96% 99% 
Mathematics 0-4 65% 92% 95% 98% 
Mathematics 0-5 65% 90% 95% 98% 
Mathematics 0-6 65% 91% 95% 97% 
ELA/L Multi-trait 65% 75% 96% 99% 


Note: ‘A 0 or 1 score compared to a blank score will have a disagreement greater than 1 point. 


Pearson’s ePEN2 scoring system included comprehensive inter-rater agreement reports that allowed 
supervisory personnel to monitor both individual and group performance. Based on reviews of these 
reports, scoring experts targeted individuals for increased backreading and feedback and, if necessary, 
retraining. Table 8.32 shows that the actual percentages for both exact reader agreement and the 
percentages of agreement within one point were higher than the inter-rater agreement expectations. 


Refer to Section 4 for more information on hand-scoring. 
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Section 9: Validity 


9.1 Overview 


The Standards for Educational and Psychological Testing, issued jointly by the American Educational 
Research Association [AERA], American Psychological Association [APA], and National Council on 
Measurement in Education [NCME] (2014) reports: 


Validity refers to the degree to which evidence and theory support the interpretations of test 
scores for proposed uses of tests. Validity is, therefore, the most fundamental consideration in 
developing tests and evaluating tests. The process of validation involves accumulating relevant 
evidence to provide a sound scientific basis for the proposed score interpretations (p. 11). 


The purpose of test validation is not to validate the test itself but to validate interpretations of the test 
scores for particular uses. Test validation is not a quantifiable property but an ongoing process, 
beginning at initial conceptualization and continuing throughout the lifetime of an assessment. Every 
aspect of an assessment provides evidence in support of its validity (or evidence of lack of validity), 
including design, content specifications, item development, and psychometric characteristics. The 2016— 
2017 operational assessment provided an opportunity to gather evidence of validity based on both test 
content and on the internal structure of the tests. 


Pearson applies the principles of universal design, as articulated in materials developed by the National 
Center for Educational Outcomes (NCEO) at the University of Minnesota (Thompson, Johnstone, and 
Thurlow, 2002). 


9.2 Evidence Based on Test Content 


Evidence based on content of achievement tests is supported by the degree of correspondence between 
test items and content standards. The degree to which the test measures what it claims to measure is 
known as construct validity. The PARCC assessments adhere to the principles of evidence-centered 
design, in which the standards to be measured (the Common Core State Standards) are identified, and 
the performance a student needs to achieve to meet those standards is delineated in the PARCC 
evidence statements. Test items are reviewed for adherence to universal design principles, which 
maximize the participation of the widest possible range of students. 


Pearson and PARCC built spreadsheets at the evidence statement level that incorporate the probability 
statements from the test blueprints and attrition rates at committee review and data review. The basis 
of our entire item development is driven by the use of these item development target spreadsheets 
provided by PARCC. Before beginning item development, Pearson uses these target spreadsheets to 
develop an internal item development plan to correlate with the expectations of the test design. These 
are reviewed and approved by PARCC. We acknowledge that each assessment has multiple parts and 
each part specifies the types of tasks and standards eligible for assessment. 
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In addition to the PARCC evidence statements, content is aligned through the articulation of 
performance in the performance level descriptors. At the policy level, the performance level descriptors 
include policy claims about the educational achievement of students who attain a particular 
performance level, and a broad description of the grade-level knowledge, skills, and practices students 
performing at a particular achievement level are able to demonstrate. Those policy-level descriptors are 
the foundation for the subject- and grade-specific performance level descriptors, which, along with the 
PARCC evidence frameworks, guide the development of the items and tasks. 


The PARCC college- and career-ready determinations (CCRD) in English language arts/literacy (ELA/L) 
and mathematics describe the academic knowledge, skills, and practices students must demonstrate to 
show readiness for success in entry-level, credit-bearing college courses and relevant technical courses. 
The PARCC states determined that this level means graduating from high school and having at least a 75 
percent likelihood of earning a grade of “C” or better in credit-bearing courses without the need for 
remedial coursework. After reviewing the standards and assessment design, the PARCC Governing Board 
(made up of the K-12 education chiefs in PARCC states) in conjunction with the PARCC Advisory 
Committee on College Readiness (composed of higher education chiefs in the PARCC states), 
determined that students who achieve at Levels 4 and 5 on the final PARCC high school assessments are 
likely to have acquired the skills and knowledge to meet the definition of college- and career-readiness. 
To validate the determinations, PARCC conducted a postsecondary educator judgment study and a 
benchmark study of the SAT, ACT, National Assessment of Educational Progress (NAEP), Trends in 
International Mathematics and Science Study (TIMSS), Programme of International Student Assessment 
(PISA), and Progress in International Reading Literacy Study (PIRLS) tests (McClarty, Korbin, Moyer, 
Griffin, Huth, Carey, and Medberry, 2015). 


Gathering construct validity evidence for PARCC is embedded in the process by which the PARCC 
assessment content is developed and validated. At each step in the assessment development process, 
PARCC states involved hundreds of educators, assessment experts, and bias and sensitivity experts in 
review of text, items, and tasks for accuracy, appropriateness, and freedom from bias. See Chapter 2 for 
an overview of the content development process. In the early stages of development, Pearson 
conducted research studies to validate the PARCC item and task development approach. One such study 
was a student task interaction study designed to collect data on the student’s experience with the 
assessment tasks and technological functionalities, as well as the amount of time needed for answering 
each task. Pearson also conducted a rubric choice study that compared the functioning of two rubrics 
developed to score the prose constructed-response (PCR) tasks in ELA/L. Quantitative and qualitative 
evidence was collected to support the use of a condensed or expanded trait scoring rubric in scoring 
student responses. 


PARCC items and tasks were field tested prior to their use on an assessment. During the initial field test 
administration in 2014, PARCC states collected feedback from students, test administrators, test 
coordinators, and classroom teachers on their experience with the PARCC assessments, including the 
quality of test items and student experience. A summary of the feedback can be found at: http://parcc- 
assessment.org/content/uploads/2017/11/PARCCCCRDPolicyandPLDsFINAL.pdf. The feedback from that 
survey was used to inform test directions, test timing, and the function of online task interactions. 
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Performance data from the field test also informed the future development of additional items and 
tasks. 


All item developers and item writers are provided an electronic version of PARCC accessibility guidelines 
and PARCC’s linguistic complexity rubric. Items and passages are reviewed internally by accessibility and 
fairness experts trained in the principles of universal design and who become well versed in PARCC’s 
accessibility guidelines. Items received internal review for alignment to PARCC evidence tables, task 
generation model, item selection guidelines, and accessibility and fairness reviews. 


An important consideration when constructing test forms is recognition of items that may introduce 
construct-irrelevant variance. Such items should not be included on test forms to help ensure fairness to 
all subgroups of test takers. PARCC convened bias and sensitivity committees to review all items. 
Additionally, content experts facilitated reviews of all items. All reviewers were trained using PARCC bias 
and sensitivity guidelines, and the guidelines were used to review items and ELA/L passages. 
Accommodations were made available based on individual need documented in the student’s approved 
IEP, 504 Plan, or if required by the PARCC member state, an English learner (EL) Plan. An accessibility 
specialist worked in consultation with the PARCC accessibility specialist to review forms and determine 
which forms should be used for students with accommodations. 


The ELA/L and mathematics operational test forms, as described in Section 2, were carefully constructed 
to align with the test blueprints and specifications that are based on the Common Core State Standards 
(CCSS). During the fall of 2016, content experts representing various PARCC states, along with other 
content experts, held a series of meetings to review the operational forms for ELA/L and mathematics. 
These meetings provided opportunity to evaluate tests forms in their entirety and recommend changes. 
Requested item replacements were accommodated to the extent possible while striving to maintain the 
integrity of the various linking designs required for the operational test analyses. Psychometricians were 
available throughout this process to provide guidance with regard to implications of item replacements 
for the linking and statistical requirements. 


Further information regarding the PARCC assessment college- and career-ready content standards, 
performance level descriptors, and accessibility features and accommodations is provided at 
http://parcc-assessment.org/college-career-ready. 


9.3 Evidence Based on Internal Structure 


Analyses of the internal structure of a test typically involve studies of the relationships among test items 
and/or test components (i.e., subclaims) in the interest of establishing the degree to which the items or 
components appear to reflect the construct on which a test score interpretation is based (AERA, APA, & 
NCME, 2014, p. 16). The term construct is used here to refer to the characteristics that a test is intended 
to measure; in the case of the PARCC operational tests, the characteristics of interest are the knowledge 
and skills defined by the test blueprint for ELA/L and for mathematics. 


The PARCC assessments provide a full summative test score, Reading claim score, and Writing claim 
score as well as ELA/L subclaims and mathematics subclaim scores. The goal of reporting at this level is 
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to provide criterion-referenced data to assess the strengths and weaknesses of a student’s achievement 
in specific components of each content area. This information can then be used by teachers to plan for 
further instruction, to plan for curriculum development, and to report progress to parents. The results 
can also be used as one factor in making administrative decisions about program effectiveness, teacher 
effectiveness, class grouping, and needs assessment. 


9.3.1 Intercorrelations 


The ELA/L full summative tests comprise two claim scores, Reading (RD) and Writing (WR), and five 
subclaim scores—Reading Literature (RL), Reading Information (RI), Reading Vocabulary (RV), Writing 
Written Expression (WE), and Writing Knowledge Language and Conventions (WKL). The RD claim score 
is a composite of RL, RI, and RV. The writing claim score, a composite of WE and WKL, comprises only 
PCR items, and the same PCR items are in each subclaim. The ELA/L operational test analyses were 
performed by evaluating the separate trait scores of WE and WKL, and for some PCR items also RL or RI; 
therefore, the trait scores were used for the intercorrelations. 


The mathematics full summative tests have four subclaim scores—Major Content (MC), Mathematical 
Reasoning (MR), Modeling Practice (MP), and Additional and Supporting Content (ASC). 


High total group internal consistencies as well as similar reliabilities across subgroups provide additional 
evidence of validity. High reliability of test scores implies that the test items within a domain are 
measuring a single construct, which is a necessary condition for validity when the intention is to 
measure a single construct. Refer to Section 8 for reliability estimates for the overall population, 
subgroups of interest, as well as for subscores for ELA/L and subclaims for mathematics. 


Another way to assess the internal structure of a test is through the evaluation of correlations among 
subscores. These analyses were conducted between the ELA/L Reading and Writing claim scores and the 
ELA/L subclaims (RL, Rl, RV, WE, and WKL) and between the mathematics subclaims. If these 
components within a content area are strongly related to each other, this is evidence of 
unidimensionality. 


A series of tables are provided to summarize the results for the spring 2017 administration.’ Tables 9.1 
through 9.9 present the Pearson correlations observed between the ELA/L Reading and Writing claim 
scores and subclaim scores for each grade; correlations are reported separately for online (CBT) and 
paper (PBT) versions of the tests. The tables provide the weighted average intercorrelations by 
averaging the intercorrelations computed for all the core operational forms of the test separately for the 
CBT and PBT tests within each grade level. The total sample size across all forms is provided in the upper 
triangle portion of the tables. The subclaim reliabilities (from Section 8) are reported along the diagonal. 
The WR, WE, and WKL scores tended to be highly correlated; this is expected given that these three 
intercorrelations are based on the trait scores from the same Writing items. RL, Rl, and RV, all subclaims 
of Reading, are moderately to highly correlated. Additionally, the WR claim and the WE and WKL 


7 Addendum 9 provides a summary of results for the fall 2016 administration. 
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subclaims are moderately correlated with RD subclaims (of RL, Rl, and RV). These moderate to high 
ELA/L intercorrelations amongst the subclaims are sufficiently high to provide evidence that the ELA/L 
tests are unidimensional. The moderate intercorrelations among the subclaims and claims suggest the 
claims may be sufficient for individual student reporting. 


The intercorrelations and reliability estimates for mathematics are provided in Tables 9.10 through 9.21. 
The mathematics intercorrelations are moderate. The only observable pattern in the mathematics 
intercorrelations is that the MC subclaim generally has slightly higher correlations with the ASC, MR, and 
MP subclaims; the intercorrelations amongst the ASC, MR, and MP subclaims are usually slightly lower. 
The mathematics intercorrelations are sufficiently high to suggest that the mathematics tests are likely 
to be unidimensional with some minor secondary dimensions. 


Additionally, the ELA/L and mathematics correlations for the two modes, PBT and CBT, displayed similar 
patterns of intercorrelations, with the CBT intercorrelations generally, but not always, being slightly 
larger than their PBT counterparts. Overall, these findings suggest that the structure of the PBT 
assessments and CBT assessments are similar. The sample sizes for Integrated Mathematics | and Ill 
PBTs were both below 100 and thus are not reported here. 
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Table 9.1 Average Intercorrelations and Reliability between Grade 3 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI RV WR. WE WKL 

RD 0.90 363,128 363,128 363,128 363,128 363,128 363,128 RD 0.89 57,188 57,188 57,188 57,188 57,188 57,188 
RL 0.93 79 363,128 363,128 363,128 363,128 363,128 RL 0.93 0.77 57,188 57,188 57,188 57,188 57,188 
RI 0.90 0.75 0.75 363,128 363,128 363,128 363,128 RI 0.90 0.73 0.71 57,188 57,188 57,188 57,188 


RV 0.88 0.74 0.71 0.73 363,128 363,128 363,128 RV 0.88 «0.73. «~0.72.—«<0.74. 57,188 57,188 57,188 
WR 0.71.~—Ss«OO.67 0.67 0.58 0.81 363,128 363,128 WR 0.75 0.70 0.70 0.63 0.8057,188 57,188 
WE 0.69 0.65 0.66 0.56 0.99 0.74 363,128 WE 0.74 0.68 0.69 0.62 0.99 0.72 57,188 
WKL 0.66 0.62 0.63 0.55 0.91 0.83 0.83 WKL 0.70 0.65 0.66 0.60 0.91 0.83 0.80 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 


Table 9.2 Average Intercorrelations and Reliability between Grade 4 ELA/L Subclaims 


CBT PBT 


RD RL Rl RV WR WE WKL RD RL Rl RV WR WE WKL 


RD 0.89 373,912 373,912 373,912 373,912 373,912 373,912 RD 0.86 48,905 48,905 48,905 48,905 48,905 48,905 
RL 0.92 0.76 373,912 373,912 373,912 373,912 373,912 RL 0.88 0.66 48,905 48,905 48,905 48,905 48,905 
Rl 0.91 0.74 0.77 373,912 373,912 373,912 373,912 Rl 0.89 0.65 0.72 48,905 48,905 48,905 48,905 
RV 0.84 0.68 0.68 0.63 373,912 373,912 373,912 RV 0.87 0.67 0.68 0.67 48,905 48,905 48,905 
WR 0.75 0.70 0.71 0.67 0.83 373,912 373,912 WR 0.76 0.70 0.68 0.62 0.79 48,905 48,905 
WE 0.74 0.69 0.70 0.56 0.99 0.80 373,912 WE 0.75 0.69 0.68 0.61 0.99 0.73 48,905 
WKL 0.70 0.65 0.66 0.54 0.93 0.88 0.84 WKL 0.70 0.65 0.62 0.58 0.92 0.87 0.78 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 
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Table 9.3 Average Intercorrelations and Reliability between Grade 5 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI RV WR WE WKL 

RD 0.87 399,223 399,223 399,223 399,223 399,223 399,223 RD 0.85 14,948 14,948 14,948 14,948 14,948 14,948 
RL 0.90 0.71 399,223 399,223 399,223 399,223 399,223 RL 0.86 0.62 14,948 14,948 14,948 14,948 14,948 
RI 0.91 0.71 0.74 399,223 399,223 399,223 399,223 RI 0.91 0.66 0.73 14,948 14,948 14,948 14,948 
RV 0.83 0.64 0.65 0.62 399,223 399,223 399,223 RV 0.82 0.59 0.63 0.5914,94814,948 14,948 
WR 0.72 0.67 0.67 0.55 0.85 399,223 399,223 WR 0.71 0.64 0.66 0.54 0.8114,948 14,948 
WE 0.71 0.66 0.66 0.54 0.99 0.79 399,223 WE 0.70 0.64 0.65 0.53 0.99 0.72 14,948 


WKL 0.71 0.65 0.65 0.55 0.95 0.90 0.86 WKL 0.68 0.61 0.63 O.51 0.94 0.89 0.80 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 


Table 9.4 Average Intercorrelations and Reliability between Grade 6 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI RV WR WE WKL 

RD 0.91 389,198 389,198 389,198 389,198 389,198 389,198 RD 0.90 17,228 17,228 17,228 17,228 17,228 17,228 
RL 0.92 0.78 389,198 389,198 389,198 389,198 389,198 RL 0.88 0.73 17,228 17,228 17,228 17,228 17,228 
RI 0.94 0.78 0.82 389,198 389,198 389,198 389,198 RI 0.94 0.75 0.81 17,228 17,228 17,228 17,228 
RV 0.87 0.72 0.74 0.70 389,198 389,198 339,198 RV 0.88 0.69 0.75 0.7117,22817,228 17,228 
WR 0.75 0.72 0.72 0.59 0.87 389,198 389,198 WR 0.74 0.72. (0.70 +0.57. 0.8417,228 17,228 
WE 0.74 0.72 0.71 0.58 1.00 0.86 389,198 WE 0.73 0.72 0.69 0.57 1.00 0.81 17,228 


WKL 0.74 0.71 0.71 0.59 0.97 0.95 0.88 WKL 0.71 0.70 0.68 0.55 0.96 0.94 0.83 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 
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Table 9.5 Average Intercorrelations and Reliability between Grade 7 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI RV WR. WE WKL 

RD 0.90 390,424 390,424 390,424 390,424 390,424 390,424 RD 0.88 15,527 15,527 15,527 15,527 15,527 15,527 
RL 0.91 0.75 390,424 390,424 390,424 390,424 390,424 RL 0.88 0.72 15,527 15,527 15,527 15,527 15,527 
RI 0.94 0.77 0.81 390,424 390,424 390,424 390,424 RI 0.92 0.70 0.75 15,527 15,527 15,527 15,527 
RV 0.84 0.67 0.70 0.64 390,424 390,424 390,424 RV 0.88 0.68 0.70 0.68 15,52715,527 15,527 
WR 0.76 0.72 0.73 0.57 0.87 390,424 390,424 WR 0.74. «0.69 «0.69 «0.58 ~=(0.84 15,527 15,527 
WE 0.75 0.71 0.73 0.56 1.00 0.86 390,424 WE 0.73 0.69 0.69 0.57 1.00 0.81 15,527 


WKL 0.75 0.71 0.73 0.57 0.97 0.95 0.89 WKL 0.72 0.67 0.67 0.57 0.97 0.94 0.82 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 


Table 9.6 Average Intercorrelations and Reliability between Grade 8 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI RV. WR. WE WKL 

RD 0.90 385,037 385,037 385,037 385,037 385,037 385,037 RD 0.89 14,113 14,113 14,113 14,113 14,113 14,113 
RL 0.93 0.79 385,037 385,037 385,037 385,037 385,037 RL 0.87 0.72 14,113 14,113 14,113 14,113 14,113 
RI 0.93 0.78 0.78 385,037 385,037 385,037 385,037 RI 0.95 0.73 0.80 14,113 14,113 14,113 14,113 
RV 0.83 0.69 0.69 0.62 385,037 385,037 385,037 RV 0.87 0.71 0.72 0.6914,113 14,113 14,113 
WR 0.77 0.74 0.73 0.59 0.88 385,037 385,037 WR 0.73. «-0.69' «0.70 «0.57. 0.8514,113 14,113 
WE 0.77 0.73 0.72 0.58 1.00 0.88 385,037 WE 0.72 0.69 0.69 0.57 1.00 0.82 14,113 


WKL 0.77 0.73 0.73 0.59 0.97 0.96 0.89 WKL 0.72 0.67 0.68 0.57 0.97 0.95 0.83 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 
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Table 9.7 Average Intercorrelations and Reliability between Grade 9 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI RV. WR WE WKL 

RD 0.90 184,918 184,918 184,918 184,918 184,918 184,918 RD 0.91 2,946 2,946 2,946 2,946 2,946 2,946 
RL 0.92 0.78 184,918 184,918 184,918 184,918 184,918 RL 0.92 0.78 2,946 2,946 2,946 2,946 2,946 
RI 0.95 0.79 0.82 184,918 184,918 184,918 184,918 RI 0.94 0.78 0.80 2,946 2,946 2,946 2,946 
RV 0.81 0.67 0.68 0.61 184,918 184,918 184,918 RV 0.87 0.73 0.74 0.69 2,946 2,946 2,946 
WR 0.78 0.75 0.75 0.56 0.89 184,918 184,918 WR 0.76 0.73 0.72 0.62 0.86 2,946 2,946 
WE 0.77 0.74 0.74 0.55 1.00 0.89 184,918 WE 0.76 0.72 0.72 061 1.00 0.83 2,946 


WKL 0.78 0.75 0.75 0.56 0.98 0.96 0.90 WKL 0.76 0.71 0.72 0.62 0.98 0.96 0.84 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 


Table 9.8 Average Intercorrelations and Reliability between Grade 10 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI. RV. WR WE. WKL 

RD 0.90 169,651 169,651 169,651 169,651 169,651 169,651 RD 0.93 1,861 1,861 1,861 1,861 1,861 1,361 
RL 0.92 0.77 169,651 169,651 169,651 169,651 169,651 RL 0.92 0.78 1,861 1,861 1,861 1,861 1,361 
RI 0.94 0.78 0.81 169,651 169,651 169,651 169,651 RI 0.96 0.82 0.86 1,861 1,8611,861 1,361 
RV 0.87 0.71 0.73 0.67 169,651 169,651 169,651 RV 0.93 0.78 0.83 0.78 1,8611,861 1,361 
WR 0.79 0.75 0.76 0.62 0.88 169,651 169,651 WR 0.82 0.80 0.79 0.70 0.871,861 1,361 
WE 0.78 0.75 0.75 0.62 1.00 0.88 169,651 WE 0.82 0.80 0.79 0.70 1.00 0.87 1,861 
WKL 0.79 0.75 0.75 0.63 0.98 0.97 0.88 WKL 0.81 0.79 0.78 0.70 0.98 0.97 0.88 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 
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Table 9.9 Average Intercorrelations and Reliability between Grade 11 ELA/L Subclaims 


CBT PBT 

RD RL RI RV WR WE WKL RD RL RI RV. WR WE. WKL 
RD 0.90 106,129 106,129 106,129 106,129 106,129 106,129 RD 0.80 1,141 1,141 1,141 1,1411,141 1,141 
RL 0.90 0.75 106,129 106,129 106,129 106,129 106,129 RL 0.84 0.61 1,141 1,141 1,1411,141 1,141 
RI 0.94 0.77 0.81 106,129 106,129 106,129 106,129 RI 0.88 0.62 0.63 1,141 1,1411,141 1,141 
RV 0.86 0.69 0.71 0.67 106,129 106,129 106,129 RV 0.79 0.54 0.52 0.48 1,1411,141 1,141 
WR 0.74 0.71 0.73 0.55 0.87 106,129 106,129 WR 0.70 0.68 0.63 0.44 0.831,141 1,141 
WE 0.74 0.71 0.72 0.55 1.00 0.86 106,129 WE 0.70 0.67 0.63 0.44 1.00 0.79 1,141 
WKL 0.74 0.70 0.72 0.55 0.98 0.97 0.86 WKL 0.69 0.67 0.63 0.43 0.99 0.97. 0.79 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = Writing, WE = Written Expression, and 
WKL = Writing Knowledge and Conventions. The shaded values along the diagonal are the reliabilities as reported in Section 8. The average 
intercorrelations are provided in the lower portion of the table and the total sample sizes are provided in the upper portion of the table. Please 
refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and subclaim. 


Table 9.10 Average Intercorrelations and Reliability between Grade 3 Mathematics Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.89 367,423 367,423 367,423 MC 0.87 58,016 58,016 58,016 
ASC 0.81 0.75 367,423 367,423 ASC 0.82 0.75 58,016 58,016 
MR 0.77 0.72 0.71 367,423 MR 0.74 0.68 0.67 58,016 
MP 0.75 0.69 0.69 0.68 MP 0.76 0.71 0.69 0.66 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 
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Table 9.11 Average Intercorrelations and Reliability between Grade 4 Mathematics Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.89 376,274 376,274 376,274 MC 0.87 49,945 49,945 49,945 
ASC 0.75 0.69 376,274 376,274 ASC 0.72 0.64 49,945 49,945 
MR 0.79 0.68 0.73. 376,274 MR 0.76 0.67 0.72 49,945 
MP 0.75 0.66 0.70 0.63 MP 0.71 0.62 0.69 0.60 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 


Table 9.12 Average Intercorrelations and Reliability between Grade 5 Mathematics Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.87. 400,272 400,272 400,272 MC 0.84 15,924 15,924 15,924 
ASC 0.73 0.63 400,272 400,272 ASC 0.71 0.62 15,924 15,924 
MR 0.78 0.66 0.72 400,272 MR 0.76 0.65 0.66 15,924 
MP 0.75 0.64 0.70 0.65 MP 0.70 0.61 0.67 0.53 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 
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Table 9.13 Average Intercorrelations and Reliability between Grade 6 Mathematics Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.87. 390,769 390,769 390,769 MC 0.83 17,527 17,527 17,527 
ASC 0.79 0.72 390,769 390,769 ASC 0.76 0.68 17,527 17,527 
MR 0.79 0.71 0.73. 390,769 MR 0.79 0.70 0.75 17,527 
MP 0.77 0.69 0.73 0.70 MP 0.71 0.65 0.71 0.64 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 


Table 9.14 Average Intercorrelations and Reliability between Grade 7 Mathematics Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.85 379,475 379,475 379,475 MC 0.76 14,182 14,182 14,182 
ASC 0.74 0.65 379,475 379,475 ASC 0.69 0.61 14,182 14,182 
MR 0.78 0.67 0.74 379,475 MR 0.70 0.63 0.72 14,182 
MP 0.76 0.66 0.71 0.54 MP 0.71 0.62 0.67 0.45 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 
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CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.81 301,671 301,671 301,671 MC 0.77 12,481 12,481 12,481 
ASC 0.73 0.63 301,671 301,671 ASC 0.68 0.58 12,481 12,481 
MR 0.74 0.66 0.70 301,671 MR 0.72 0.61 0.62 12,481 
MP 0.71 0.65 0.68 0.60 MP 0.69 0.62 0.65 0.58 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 


information about the number of items and number of score points in each claim and subclaim. 


Table 9.16 Average Intercorrelations and Reliability between Algebra | Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.82 252,814 252,814 252,814 MC 0.78 6,364 6,364 6,364 
ASC 0.79 0.72 252,814 252,814 ASC 0.75 0.63 6,364 6,364 
MR 0.78 0.73 0.74 252,814 MR 0.70 0.65 0.65 6,364 
MP 0.77 0.71 0.73 0.75 MP 0.74 0.68 0.68 0.70 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 


information about the number of items and number of score points in each claim and subclaim. 
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Table 9.17 Average Intercorrelations and Reliability between Geometry Subclaims 


CBT PBT 

MC ASC MR MP MC ASC MR MP 
MC 0.86 135,546 135,546 135,546 MC 0.87 1,482 1,482 1,482 
ASC 0.82 0.76 135,546 135,546 ASC 0.78 0.71 1,482 1,482 
MR 0.75 0.70 0.74 135,546 MR 0.68 0.63 0.66 1,482 
MP 0.77 6:72 0.76 0.68 MP 0.78 0.68 0.64 0.71 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 


table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 


Table 9.18 Average Intercorrelations and Reliability between Algebra II Subclaims 


CBT PBT 

MC ASC MR MP MC ASC MR MP 

MC 0.82 119,653 119,653 119,653 MC 0.80 2,279 2,279 2,279 
ASC 0.81 0.77 119,653 119,653 ASC 0.79 0.74 2,279 2,279 
MR 0.74 0.74 0.70 119,653 MR 0.73 0.72 0.68 2,279 
MP 0.73 0.73 0.71 0.65 MP 0.74 0.72 0.69 0.66 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 


table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 
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Table 9.19 Average Intercorrelations and Reliability between Integrated Mathematics | Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.77 10,842 10,842 10,842 MC 
ASC 0.80 0.77 10,842 10,842 ASC 
MR 0.76 0.76 0.69 10,842 MR 
MIP 0.70 0.70 0.72 0.73 MP 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Integrated Mathematics | PBT had insufficient sample sizes. 
Please refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and 
subclaim. 


Table 9.20 Average Intercorrelations and Reliability between Integrated Mathematics II Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MP 
MC 0.78 1,999 1,999 1,999 MC 0.62 196 196 196 
ASC 0.68 0.65 1,999 1,999 ASC 0.67 0.63 196 196 
MR 0.67 0.57 0.57 1,999 MR 0.69 0.63 0.48 196 
MP 0.67 0.58 0.54 0.62 MP 0.65 0.64 0.65 0.59 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Please refer to Appendix A.12.1 (Form Composition) for 
information about the number of items and number of score points in each claim and subclaim. 


March 27, 2018 Page 149 


PARCC 2017 Technical Report 


Table 9.21 Average Intercorrelations and Reliability between Integrated Mathematics III Subclaims 


CBT PBT 
MC ASC MR MP MC ASC MR MIP 
MC 0.83 500 500 500 MC 
ASC 0.82 0.73 500 500 ASC 
MR 0.76 0.73 0.74 500 MR 
MIP 0.73 0.72 0.73 0.69 MP 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, and MP = Modeling Practice. The shaded 
values along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are provided in the lower portion of the 
table and the total sample sizes are provided in the upper portion of the table. Integrated Mathematics III PBT had insufficient sample sizes. 
Please refer to Appendix A.12.1 (Form Composition) for information about the number of items and number of score points in each claim and 
subclaim. 
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9.3.2 Reliability 


Additionally, the reliability analyses presented in Section 8 of this technical report provide information 
about the internal consistency of the PARCC assessments. Internal consistency is typically measured via 
correlations amongst the items on an assessment and provides an indication of how much the items 
measure the same general construct. The reliability estimates, computed using coefficient alpha 
(Cronbach, 1951), are presented in Tables 8.1 and 8.2 and are along the diagonals of Tables 9.1 through 
9.21.8 The average reliabilities for ELA/L PARCC assessments and for the mathematics assessments range 
from .86 up to .94. Tables 8.3 through 8.11 summarize test reliability for groups of interest for ELA/L 
grades 3 through 11, and Tables 8.12 through 8.23 summarize test reliability for groups of interest for 
mathematics grades/subjects. Along with the subclaim intercorrelations, the reliability estimates 
indicate that the items within each PARCC assessment are measuring the same construct and provides 
further evidence of unidimensionality. 


9.3.3 Local Item Dependence 


In addition to the intercorrelations for ELA/L and mathematics, local item independence was evaluated. 
Local independence is one of the primary assumptions of IRT that states the probability of success on 
one item is not influenced by performance on other items, when controlling for ability level. This implies 
that ability or theta accounts for the associations among the observed items. Local item dependence 
(LID) when present essentially overstates the amount of information predicted by the IRT model. It can 
exert other undesirable psychometric effects and represents a threat to validity since other factors 
besides the construct of interest are present. Classical statistics are also affected when LID is present 
since estimates of test reliability like IRT information can be inflated (Zenisky, Hambleton, & Sireci, 
2003). 


The LID issue affects the choice of item scoring in IRT calibrations. Specifically, if evidence suggests these 
items indeed have local dependence, then it might be preferable to sum the item scores into clusters or 
testlets as a method of minimizing LID. However, if these items do not appear to have strong local item 
dependence, then retaining the scores as individual item scores in an IRT calibration is preferred since 
more information concerning item properties is retained. During the initial operational administration of 
the PARCC assessments in spring 2015, a study that included two methods of investigating the presence 
of LID was conducted. A description of the methods along with study findings are summarized below. 


First, analyses of the internal consistency in items and testlets were conducted under classical test 
theory (Wainer & Thissen, 2001) as a way to evaluate the degree of LID. Two estimates of Cronbach’s 
alpha (Cronbach, 1951) were compared based on individual items in a test and those clustered into 
testlets. Cronbach’s alpha is formulated as: 


8 Section 8 provides information on the computations of the reliability estimates. 
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k-1 o, ; (9-1) 


where & is the total number of items, <, is the covariance of items 7 and i' (i # i'), and o, is the 
variance of total scores. To compute an alpha coefficient, sample standard deviations and variances are 
substituted for the «, and Oo. . The alpha for the total test based on individual items is compared with 


those that form testlets based on larger subparts. If the item-level configuration has appreciably higher 
levels of internal consistency compared with the testlets, LID may be present. 


For IRT-based methods, local dependence can be evaluated using statistics such as Q3 (Yen, 1984). The 
item residual is the difference between observed and expected performance. The Q; index is the 
correlation between residuals of each item pair defined as 


d,=(O-E), 


Q,=7(4,.d,) 


(9-2) 


where O is the observed score and £ is the expected value of O under a proposed IRT model and 
the index is defined as the correlation between the two item residuals. 


LID manifests itself as a residual correlation that is nonzero and large. For Q;, LID can be either positive 
or negative. Positive (negative) LID indicates that performance is higher (lower) than expectation. The 
residual Q3;correlation matrix can be inspected to determine if there are any blocks of locally dependent 
items (e.g., perhaps blocks of items belonging to the same reading passage). For Q;3, the null hypothesis 
is that local independence holds. The expected value of Q; is -1/n-1 where n is the number of items such 
that the statistic shows a small negative bias. As a rule of thumb, item pairs with moderate levels of LID 
for Q3 are |0.20| or greater. Significant levels of LID are present when the statistic is greater than 
|0.40|. An alternative is to use the Fisher r to z transformation and evaluate the resulting p values. 


For the LID comparisons, the following eight test levels administered in spring 2015 were selected: 


Grade 4 for span 3-5 in ELA/L, 

Grade 4 for span 3-5 in mathematics, 

Grade 7 for span 6-8 in ELA/L, 

Grade 7 for span 6-8 in mathematics, 

Grade 10 for span 9-11 in ELA/L, 

Integrated Mathematics II for Integrated Mathematics I-III, 
Algebra |, and 

Algebra Il. 


00: OOF a: Ne 


One spring 2015 CBT form for each of the eight tests was selected that was roughly at the median in 
terms of test difficulty. For ELA/L, reading items were summed according to passage assignment. For 
mathematics, items were summed according to subclaims. Cronbach’s alpha was computed for the 
entire forms using the two different approaches as described above, one involving calculations at the 
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item level and the second utilizing scores on summed items (i.e., testlets). Further description of the 
data is given in Table 9.22. 


To cross-validate the internal consistency analysis, the Q; statistic was computed from spring CBT data 
based on grade 4 ELA/L and Integrated Mathematics II items. All items in the pool at that test level were 
included. The CBT item pool for grade 4 ELA/L contained 125 items while Integrated Mathematics II had 
77 items. 


The results for the internal consistency analysis are shown in Figure 9.1. In every instance, the item-level 
Cronbach's alpha is higher than in the testlet configuration. The greatest difference was for Algebra II, 
which showed a difference of 0.07. Although this was not unexpected, the magnitude of the differences 
in the respective alpha coefficients in general do not suggest a concerning level of LID. Table 9.23 shows 
the summary for the Q;values. Figures 9.2 and 9.3 show graphs of the distribution of Q3values. Most of 
the Q;3 values were small and negative, again suggesting that LID is not at a level of concern. 


For these two test levels, the difference in the alpha coefficients was 0.03 and was consistent with the 
low values of Q3. 


In summary, this investigation did not find evidence for the existence of pervasive LID. The results of 
both the internal consistency analyses and Q3 methods support a claim of minimal LID. For a multiple- 
choice-only test containing four reading passages with 5 to 12 items associated with a reading passage, 
Sireci, Thissen, and Wainer (1991) reported that testlet alpha was approximately 10 percent lower than 
the item-level coefficient. In comparison, PARCC tests have complex test structures and exhibited 
smaller differences in alpha coefficients. In addition, the median Q;3 values presented in Table 9.23 
centered around the expectation of -1/n-1. 


£ 
2 0.85 
«<I 
- HI | 


ELAOS ELAO?7 ELA10 MATOS 7 ALGO1 ALGO2 MAT2 
ELA/L Mathematics 


@item Reliability m Clustered Reliability 


Figure 9.1 Comparison of Internal Consistency by Item and Cluster (Testlet) 
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Table 9.22 Conditions used in LID Investigation and Results 


N N Percent No. No. Item Task 
Content Grade Valid Complete Incomplete Items Tasks _ Rel. Rel. 
ELA/L 
ELA/L 4 13,660 13,518 1.04 31 5 0.86 0.83 
ELA/L 7 12,757 12,685 0.56 41 7 0.89 0.88 
ELA/L 10 3,097 3,033 2.07 41 7 0.90 0.87 


Mathematics 


Math 4 10,332 10,255 0.75 53 4 0.93 0.92 
Math 7 10,295 10,188 1.04 50 6 0.92 0.87 
Math Al 5,072 4,885 3.69 52 6 0.90 0.85 
Math A2 4,982 4,769 4.28 54 6 0.92 0.85 
Math M2 2,708 2,645 2.33 51 6 0.90 0.87 


Note: A1 = Algebra |, A2 = Algebra Il, M2 = Integrated Mathematics II. 


Table 9.23 Summary of Q3 Values for ELA/L Grade 4 and Integrated Mathematics II (Spring 2015) 


Min. Q: Median Mean Q3 Max. SD 
ELA/L Grade 4 
-0.138 -0.047 -0.031 -0.031 -0.017 0.279 0.030 
Integrated Mathematics II 
-0.160 -0.038 -0.017 -0.019 0.001 0.280 0.032 
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Figure 9.2 Distribution of Q; Values for Grade 4 ELA/L (Spring 2015) 
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Figure 9.3 Distribution of Q; Values for Integrated Mathematics II (Spring 2015) 
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9.4 Evidence Based on Relationships to Other Variables 


Empirical results concerning the relationships between scores on a test and measures of other variables 
external to the test can also provide evidence of validity when these relationships are found to be 
consistent with the definition of the construct that the test is intended to measure. As indicated in the 
AERA, APA, and NCME standards (2014), the variables investigated can include other tests that measure 
the same construct and different constructs, criterion measures that scores on the test are expected to 
predict, as well as demographic characteristics of test takers that are expected to be related and 
unrelated to test performance. 


The relationship of the scores across the ELA/L and mathematics assessments was evaluated using 
correlational analyses. Tables 9.24 through 9.29 present the Pearson correlations observed between the 
ELA/L scale scores and the mathematics scale scores for each grade; the correlations are reported 
separately for online (CBT) and paper (PBT) versions of the tests. For grades 3 through 8, students must 
have a valid test score for both ELA/L and mathematics at the same grade level to be included in the 
tables. These tables provide the correlation in the lower triangle and the sample size is provided in the 
upper triangle. In computing the correlations between a particular pair of ELA/L and mathematics tests, 
test takers must have taken both tests via the same mode in spring 2017. Across both modes of ELA/L, 
Reading (RD), and Writing (WR) are moderately to highly correlated with mathematics; the correlations 
range from .60 up to .80 for grades 3 through 8. These correlations suggest that the ELA/L and 
mathematics tests are assessing different content. The higher intercorrelations between the ELA/L, 
Reading (RD), and Writing (WR) suggest stronger internal relationships when compared to the 
correlations with the mathematics content area. 


The ELA/L and mathematics correlations for the high school tests are presented in Tables 9.30 through 
9.32. Because students in high school can take the mathematics courses in different years (e.g., one 
student make take Algebra | in grade 9 while another student may take Algebra | in grade 10), the high 
school mathematics scores were correlated with several of the ELA/L grades (e.g., Algebra | correlated 
with both grades 9 and 10). Only correlations for pairings with total sample sizes of at least 100 are 
shown in the tables. Across both modes of grades 8 through 11 ELA/L, Reading (RD) and Writing (WR) 
claims have correlations with high school mathematics tests that range from .22 to .75. Correlations 
between high school mathematics scores and corresponding ELA/L scores demonstrate low to moderate 
correlations. 
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CBT PBT 
ELA/L RD WR MA ELA/L RD WR MA 
ELA/L 360,980 360,980 360,980 ELA/L 56,195 56,195 56,195 
RD 0.96 360,980 360,980 RD 0.96 56,195 56,195 
WR 0.85 0.71 360,980 WR 0.89 0.74 56,195 
MA 0.79 0.78 0.68 MA 0.77 0.75 0.68 


Note: ELA/L = English language arts/literacy, RD = Reading, WR = Writing, MA = Mathematics. The correlations are provided in the lower portion 
of the table and the sample sizes are provided in the upper portion of the table. Students must have a valid grade 3 ELA/L score and a valid grade 


3 mathematics score to be included in this table. 


Table 9.25 Correlations between ELA/L and Mathematics for Grade 4 


CBT PBT 
ELA/L RD WR MA ELA/L RD WR MA 
ELA/L 371,799 371,799 371,799 ELA/L 48,153 48,153 48,153 
RD 0.96 371,799 371,799 RD 0.96 48,153 48,153 
WR 0.89 0.74 371,799 WR 0.90 0.75 48,153 
MA 0.80 0.79 0.69 MA 0.76 0.76 0.65 


Note: ELA/L = English language arts/literacy, RD = Reading, WR = Writing, MA = Mathematics. The correlations are provided in the lower portion 
of the table and the sample sizes are provided in the upper portion of the table. Students must have a valid grade 4 ELA/L score and a valid grade 


4 mathematics score to be included in this table. 
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CBT PBT 
ELA/L RD WR MA ELA/L RD WR MA 
ELA/L 396,828 396,828 396,828 ELA/L 14,516 14,516 14,516 
RD 0.95 396,828 396,828 RD 0.94 14,516 14,516 
WR 0.86 0.71 396,828 WR 0.88 0.70 14,516 
MA 0.77 0.75 0.65 MA 0.74 0.74 0.62 


Note: ELA/L = English language arts/literacy, RD = Reading, WR = Writing, MA = Mathematics. The correlations are provided in the lower portion 
of the table and the sample sizes are provided in the upper portion of the table. Students must have a valid grade 5 ELA/L score and a valid grade 


5 mathematics score to be included in this table. 


Table 9.27 Correlations between ELA/L and Mathematics for Grade 6 


CBT PBT 
ELA/L RD WR MA ELA/L RD WR MA 
ELA/L 385,764 385,764 385,764 ELA/L 15,675 15,675 15,675 
RD 0.96 385,764 385,764 RD 0.95 15,675 15,675 
WR 0.86 0.72 385,764 WR 0.87 0.71 15,675 
MA 0.80 0.80 0.66 MA 0.79 0.79 0.64 


Note: ELA/L = English language arts/literacy, RD = Reading, WR = Writing, MA = Mathematics. The correlations are provided in the lower portion 
of the table and the sample sizes are provided in the upper portion of the table. Students must have a valid grade 6 ELA/L score and a valid grade 


6 mathematics score to be included in this table. 
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CBT PBT 
ELA/L RD WR MA ELA/L RD WR MA 
ELA/L 374,078 374,078 374,078 ELA/L 12,388 12,388 12,388 
RD 0.95 374,078 374,078 RD 0.95 12,388 12,388 
WR 0.89 0.74 374,078 WR 0.88 0.71 12,388 
MA 0.78 0.78 0.66 MA 0.74 0.75 0.60 


Note: ELA/L = English language arts/literacy, RD = Reading, WR = Writing, MA = Mathematics. The correlations are provided in the lower portion 
of the table and the sample sizes are provided in the upper portion of the table. Students must have a valid grade 7 ELA/L score and a valid grade 


7 mathematics score to be included in this table. 


Table 9.29 Correlations between ELA/L and Mathematics for Grade 8 


CBT PBT 
ELA/L RD WR MA ELA/L RD WR MA 
ELA/L 296,823 296,823 296,823 ELA/L 10,975 10,975 10,975 
RD 0.95 296,823 296,823 RD 0.95 10,975 10,975 
WR 0.89 0.72 296,823 WR 0.91 0.73 10,975 
MA 0.74 0.75 0.62 MA 0.78 0.78 0.65 


Note: ELA/L = English language arts/literacy, RD = Reading, WR = Writing, MA = Mathematics. The correlations are provided in the lower portion 
of the table and the sample sizes are provided in the upper portion of the table. Students must have a valid grade 8 ELA/L score and a valid grade 


8 mathematics score to be included in this table. 
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CBT PBT 
ELA/L ELA/L 
Al GO A2 M1 M2 M3 Al GO A2 M1 M2 M3 
8 0.68 0.56 0.61 0.58 0.47 8 0.66 0.47 
(71,566) (10,548) (602) (1,413) (157) (804) (117) 
9 0.67 0.68 0.70 0.73 0.62 0.59 9 0.71 0.59 
(115,968) (39,003) (8,521) (8,551) (1,458) (191) (2,259) (363) 
10 0.54 0.62 0.66 0.68 0.61 10 0.55 0.57 0.74 
(11,319) (68,746) (35,784) (409) (219) (180) (511) = (192) 
11 0.45 0.50 0.57 0.56 0.73 0.67 11 0.47 0.62 0.64 
(2,410) (8,915) (57,832) (248) (113) (323) (106) (594) (169) 


Note: ELA/L = English language arts/literacy, Al = Algebra 1, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics Il, M3 = Integrated Mathematics III. The correlations are provided with the sample sizes, below in parentheses. Shaded cells 


indicate pairings with sample sizes less than 100. 


Table 9.31 Correlations between ELA/L Reading and Mathematics for High School 


- CBT ™ PBT 
Al GO A2 M1 M2 M3 Al GO A2 M1 M2 M3 

8 0.69 0.57 0.61 0.61 0.42 8 0.69 0.59 

(71,566) (10,548) (602) (1,413) (157) (804) (117) 
9 0.67 0.69 0.71 0.74 0.64 0.60 9 0.71 0.57 

(115,968) (39,003) (8,521) (8,551) (1,468) (191) (2,259) (363) 
10 0.54 0.63 0.66 0.67 0.60 10 0.54 0.55 0.75 

(11,319) (68,746) (35,784) (409) (219) (180) (511) = (192) 
11 0.46 0.51 0.59 0.57 0.72 0.70 11 0.43 0.61 0.66 

(2,410) (8,915) (57,832) (248) (113) (323) (106) (594) (169) 


Note: RD = Reading, A1 = Algebra |, GO = Geometry, A2 = Algebra Il, M1 = Integrated Mathematics |, M2 = Integrated Mathematics II, M3 = 
Integrated Mathematics III. The correlations are provided with the sample sizes, below in parentheses. Shaded cells indicate pairings with 


sample sizes less than 100. 
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CBT PBT 
WR WR 
Al GO A2 M1 M2 M3 Al GO A2 M1 M2 M3 
8 0.57 0.44 0.50 0.42 0.38 8 0.46 0.22 
(71,566) (10,548) (602) (1,413) (157) (804) (117) 
9 0.55 0.55 0.58 0.57 0.47 0.45 9 0.58 0.49 
(115,968) (39,003) (8,521) (8,551) (1,468) (191) (2,259) (363) 
10 0.45 0.52 0.56 0.56 0.50 10 0.46 0.48 0.65 
(11,319) (68,746) (35,784) (409) (219) (180) (511) ~— (192) 
4 0.36 0.38 0.44 0.37 0.63 0.49 11 0.42 0.53 0.54 
(2,410) (8,915) (57,832) (248) (113) (323) (106) (594) (169) 


Note: WR = Writing, A1 = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated Mathematics II, M3 = 
Integrated Mathematics III. The average correlations are provided with the sample sizes, below in parentheses. Shaded cells indicate pairings 


with sample sizes less than 100. 
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9.5 Evidence from the Special Studies 


Several research studies were conducted to provide additional validity evidence for the PARCC’s goals of 
assessing more rigorous academic expectations, helping to prepare students for college and careers, and 
providing information back to teachers and parents about their students’ progress toward college and 
career readiness. Some of the special studies conducted include: 


e content alignment studies, 

e abenchmarking study, 

e =a longitudinal study of external validity 
e amode comparability study, and 

e adevice comparability study. 


The following paragraphs briefly describe each of these studies. 


9.5.1 Content Alignment Studies 


In 2016, content of the ELA/L assessments at grades 5, 8, and 11 and the Algebra II and Integrated 
Mathematics Il assessments were evaluated to determine how well the PARCC assessments were 
aligned to the Common Core State Standards (CCSS; Doorey, & Polikoff, 2016, Schultz, Michaels, Dvorak, 
& Wiley, 2016). These content alignment studies were conducted by the Fordham Institute for grades 5 
and 8 and by Human Resources Research Organization (HumRRO) for the high school assessments. Both 
of these studies used the same methodology by having content experts review the assessment items 
and answers (for the constructed-response items the rubrics were reviewed). The content experts then 
judged how well the items aligned to the CCSS, the depth of knowledge of the items, and the 
accessibility of the items to all students, including English learners and students with disabilities. The 
authors of both studies noted that the content experts reviewing the assessments were required to be 
familiar with the CCSS but could not be employed by participating organizations or be the writers of the 
CCSS. Therefore, an effort was made to eliminate any potential conflicts of interest. 


The content studies had the individual content experts review and rate each item; then as a group the 
content experts came to a consensus on the final ratings for the content alignment, depth of knowledge, 
and accessibility to all students. In addition to the ratings, the content experts were asked to make 
comments that provided an explanation of their ratings; these comments were then used by the full 
group of content experts to provide narrative comments regarding the overall ratings and to provide 
feedback and recommendation about the assessment programs. 


The PARCC assessment program was rated as Excellent Match for ELA/L content and depth and Good 
Match for mathematics content and depth for grades 5 and 8. However, for grade 11 ELA/L content was 
rated as Excellent Match but depth was rated as Limited/Uneven Match. The high school mathematics 
assessments were rated at Excellent Match for content and Good Match for depth. 


The content studies noted some weaknesses and strengths of the PARCC assessments. For ELA/L, it was 
noted that the assessments include complex texts, a range of cognitive demands, and have a variety of 


March 27, 2018 Page 162 


PARCC 2017 Technical Report 


item types. Furthermore, the ELA/L “assessments require close reading, assess writing to sources, 
research, and inquiry, and emphasize vocabulary and language skills” (Doorey & Polikoff, 2016). The 
grade 11 ELA/L assessment had a smaller range of depth and included items assessing the higher- 
demand cognitive level. A weakness of the ELA/L assessments is the lack of a listening and speaking 
component. It was also suggested that the ELA/L assessments could be enhanced by the inclusion of a 
research task that requires the use of two or more sources of information. 


The strengths of the mathematics assessments include assessments that are aligned to the major work 
for each grade level. While the grade 5 assessment includes a range of cognitive demand, the grade 8 
assessment includes a number of higher-demand items and may not fully assess the standards at the 
lowest level of cognitive demand. It was suggested that the grade 5 assessment could include more 
focus on the major work and the grade 8 assessment could include items at the lowest cognitive 
demand level. Additionally, the reviewers noted that some of the mathematics items should be carefully 
reviewed for editorial and mathematical accuracy. 


The high school report noted that the PARCC assessment program incorporates a number of accessibility 
features and test accommodations for students with disabilities and for English learners. Furthermore, 
the PARCC assessments included items designed to accommodate the needs of students with 
disabilities. 


In 2017, HumRRO conducted a study to evaluate the quality and alignment of ELA/L and mathematics 
assessments for grades 3, 4, 6, and 7 (Shultz, Norman-Dvorak, Chen, 2017). This alignment study 
followed a similar methodology as the 2016 study. Reviewers were asked to determine the extent to 
which items were aligned to the CCSS, using fully, partially, or not aligned as the rating categories. 
Ratings were averaged to determine overall alignment. For ELA/L, 99.6 percent of grade 3 and 4 items, 
95.5 percent of grade 6 items, and 94.6 percent of grade 7 items were fully aligned. For mathematics, 
92.0 percent of grade 3, 91.1 percent of grade 4 items, 83.1 percent of grade 6 items, and 94.0 
percent of grade 7 items were fully aligned. The majority of the items that did not fall into fully aligned 
were considered partially aligned to the standards. CCSS are designed to be measured by multiple items, 
so items that aligned to multiple CCSS received a partially aligned rating. The overall item to CCSS 
alignment was captured by a holistic alignment rating which indicated if an item captured the identified 
standards as a set. Holistic ratings (either yes or no) were found by averaging review ratings across 
clusters for items that included more than one standard. For ELA, for all four grades, at least 93 percent 
of items had a holistic alignment rating of yes to indicate that the identified standards captured the skills 
or knowledge required. For mathematics, grade 6 had the lowest percentage for the holistic alignment 
rating of yes (84.8 percent), and grade 7 had the highest (96.3 percent). Overall the alignment study 
suggests that the identified CCSS capture the knowledge and skills required in the items. 


In addition to the alignment study, HumRRO also evaluated the CCSSO criteria for content and depth for 
ELA/L and mathematics grades 3, 4, 6 and 7, as well as the cognitive complexity levels of these same 
grades. There are five criteria for ELA/L content: close reading, writing, vocabulary and language skills, 
research and inquiry, and speaking and listening. Reviewers were asked to rate the content as Excellent, 
Good, Limited/Uneven, or Weak Match. For grades 3, 4, 5, and 7, the ELA/L assessments received a 
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composite rating of Excellent Match for assessing the content needed for college and career readiness. 
There are four criteria for ELA/L depth: text quality and types, complexity of texts, cognitive demand, 
and high-quality items and item variety. All grades in this study received a composite rating of Good 
Match for depth. For mathematics content, the composite rating is based on two criteria: focus and 
concepts, procedures, and applications. Grades 3, 4, and 6 received a composite content rating of Good 
Match, and grade 7 received a composite content rating of Excellent Match. The mathematics composite 
depth rating is based on three criteria: connecting practice to content, cognitive demand, and high- 
quality items and item variety. All grades in the study were rated as Excellent Match at assessing the 
depth needed to successfully meet college and career readiness. 


Finally, the 2017 HumRRO study looked at cognitive complexity of the items on ELA/L and mathematics 
at grades 3, 4, 6, and 7. Reviewers indicated their agreement with the intended cognitive complexity 
ratings provided by PARCC of low, medium, or high. The results indicated that the reviewers generally 
agreed with the distribution of complexity levels assigned by PARCC. There were differences in 
agreements in ELA/L language cluster and a few exceptions to agreement in math, particularly at grade 
6, where there was disagreement in the ratings at the medium complexity level for two domains and the 
high complexity level for one domain. For grade 7, there was agreement across low, medium, and high 
in all domains. 


9.5.2 Benchmarking Study 


The purpose of the PARCC benchmarking study (McClarty, Korbin, Moyer, Griffin, Huth, Carey, and 
Medberry, 2015) was to provide information that would inform the PARCC performance level setting 
(PLS) process. PARCC used an evidence-based standard setting approach (EBSS; McClarty, Way, Porter, 
Beimers, & Miles, 2013) to establish the performance levels for its assessments. In EBSS, the threshold 
scores for performance levels are set based on a combination of empirical research evidence and expert 
judgment. This benchmarking study provided one source of empirical evidence to inform the PARCC 
college- and career-readiness performance level (i.e., Level 4). The study findings were provided to 
PARCC’s pre-policy standard-setting committee. The charge of this committee was to suggest a 
reasonable range for the percentage of students meeting or exceeding the PARCC Level 4 threshold 
score and therefore considered college- and career-ready. Section 11.3.2 of this report provides more 
information about the PARCC pre-policy meeting. For the PARCC benchmarking study, external 
information was analyzed to provide information about the Level 4 threshold scores for the grade 11 
ELA/L, Algebra II, and Integrated Mathematics II assessments, the grade 8 ELA/L and mathematics 
assessments, and the grade 4 ELA/L and mathematics assessments. The PARCC assessments and Level 4 
expectations were compared with comparable assessments and expectations for the Programme of 
International Student Assessment (PISA), Trends in International Mathematics and Science Study 
(TIMSS), Progress in International Reading Literacy Study (PIRLS), National Assessment of Educational 
Progress (NAEP), ACT, SAT, the Michigan Merit Exam, and the Virginia End-of-Course exams. For each 
external assessment, the best-matched performance level was determined and the percentage of 
students reaching that level across the nation and in the PARCC states was determined. Across all grades 
and subjects, the data indicated approximately 25 to 50 percent of students were college- and career- 
ready or on track to readiness based on PARCC’s Level 4 expectations. 
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For details on how the benchmarking study was used during the standard setting process, refer to 
Section 11 of this technical report. 


9.5.3 Longitudinal Study of External Validity of PARCC Performance Levels (Phase 1) 


In 2016-2017, the first phase of a two-part external validity study of claims about the alignment of 
PARCC Level 4 to college readiness was completed (Steedle, Quesen, and Boyd, 2017) using PARCC 
assessment scores from the 2014—2015 and 2015-2016 academic years. Associations between PARCC 
performance levels and college-readiness benchmarks established by the College Board and ACT were 
used to study the claim that students who achieve Level 4 have a .75 probability of attaining at least a C 
in entry-level, credit-bearing, postsecondary coursework. Regression estimates measured the 
relationship between the PARCC and external test scores. The PARCC Level 4 benchmark was used to 
estimate the expected score on an external test, and vice versa. Assessment scores were dichotomized 
for additional analyses. Cross-tabulation tables provided classification agreement among tests. Logistic 
regression modeled the relationship between students’ PARCC scores and their probabilities of meeting 
the external assessment benchmark, and vice versa. 


These methods were used to make the following comparisons in mathematics: Algebra | and PSAT10 
Math; Geometry and PSAT10 Math; Algebra Il and PSAT10 Math; Algebra II and PSAT/NMSQT Math; 
Algebra Il and SAT Math; and Algebra II and ACT Math. The classification agreement (meeting the 
benchmark on both tests or not meeting the benchmark on both tests) ranged from 62.5 percent to 86.5 
percent. The overall trend indicated that students who met the benchmark on a PARCC mathematics 
assessment were likely to meet or exceed the benchmark on an external test (probabilities ranged from 
.509 to .886). However, students who met the benchmark on the external test had relatively low 
probabilities of meeting the PARCC mathematics benchmark (.097 to .310). 


The following comparisons were made in English language arts/literacy: grade 9 and PSAT10 evidence- 
based reading and writing (EBRW); grade 10 and PSAT10 EBRW;; grade 10 and PSAT/NMSQT EBRW; 
grade 10 and SAT EBRW;; grade 11 and PSAT/NMSQT EBRW;; grade 11 and SAT EBRW; grade 11 and ACT 
English; and grade 11 and ACT reading. In the majority of comparisons, the trend in ELA/L results was 
similar to mathematics. The classification agreements ranged from 67.3 percent to 79.7 percent. 
Students meeting the PARCC ELA/L benchmark had probabilities between .667 and .825 of meeting the 
benchmark on the external assessment. However, a student taking the external test had lower 
probabilities of meeting the benchmark on the PARCC ELA/L assessments (.326 to .513). 


Overall, results indicated that a student meeting the benchmark on the PARCC test had a high 
probability of making the benchmark on the external test, but the converse did not hold for students 
meeting the benchmark on the external test, for the majority of comparisons. These results suggest that 
meeting the PARCC benchmark is an indicator of academic readiness for college. However, it may be 
that students who meet the PARCC benchmark have a greater than .75 probability of earning a C or 
higher in first-year college courses. 


Phase 1 is a preliminary study using indirect comparisons; therefore, there are limitations to 
interpretations. Phase 2 of this study (to occur in 2018) will use longitudinal data including academic 
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performance in entry-level college courses for students who took PARCC assessments during high 
school. This second phase may lend additional support for the validity of the PARCC Level 4 performance 
level as a predictor of postsecondary academic success. 


9.5.4 2017 Mode and Device Comparability Studies 


PARCC assessments have been operational since the 2014—2015 school year. In addition to the 
traditional paper format, the assessments were available for online administration via a variety of 
electronic devices, including desktop computers, laptop computers, and tablets. PARCC’s research 
agenda includes several studies evaluating the interchangeability of scale scores across modes and 
devices. 


This report describes a two-pronged study consisting of a mode comparability analysis and a device 
comparability analysis. In the mode comparability analysis, scores arising from the paper administration 
were compared to those arising from any type of online administration. In the device comparability 
analysis, online scores arising from tests administered using a tablet are compared with online scores 
arising from any other type of electronic administration where a tablet was not present (i.e., laptops, 
desktops, Chromebooks). 


The goal of this study was threefold: 1) to investigate whether assessment items were of similar 
difficulty across the levels of conditions for each analysis (i.e., paper and online for the mode 
comparability analysis and tablet and non-tablet for the device comparability analysis); 2) to determine 
whether the psychometric properties of test scores were similar across the levels of conditions for each 
analysis; and 3) to determine whether overall test performance was similar across the levels of 
conditions for each analysis. 


This study examined performance on 12 PARCC assessments, split evenly between mathematics and 
ELA/L. Students were matched on demographic variables as well as the score from the PARCC 
assessment in the same content area in the prior year, creating comparable samples that allowed for an 
unbiased comparison of performance across different conditions. 


The results of the mode comparability analysis were mixed and found to be consistent with prior 
research. The item means suggested that items were of similar difficulty on paper and online modes. 
Only two items were flagged for mode effects, both of which were on the mathematics assessments. C- 
level differential item functioning (DIF) was present in both analyses. All the items flagged for C-level DIF 
in the mathematics assessments favored the online students, whereas the majority items flagged for C- 
level DIF in the ELA/L assessments favored the paper students. An examination of test reliability 
displayed comparable reliability values between the two modes; none of the test forms were flagged for 
mode effects with respect to test reliability. The test-level adjustment analysis as well as the change of 
the paper students’ performance levels after the adjustment constants were applied to the paper 
students’ scores indicated that more scale scores were adjusted downward than were adjusted upward 
on the paper test form for each assessment except grades 5 and 7 mathematics. However, all 
adjustments were less than the minimum standard error of Theta except for grade 11 ELA/L, which was 
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the same as the minimum standard error of Theta. Therefore the adjustments are within measurement 
precision for each assessment. 


The results of the device comparability study revealed consistent evidence supporting the comparability 
between the tablet condition (TC) and the non-tablet condition (NTC). Specifically, the item means 
suggested that items were similarly difficult for the TC and NTC, and none of the items were flagged for 
device effects. The DIF analysis revealed that none of the items had C-level DIF. Consistent with the 
findings at the item level, an examination of test reliability indicated that the TC and NTC test forms 
were similarly reliable and that none of the test forms were flagged for device effects. Furthermore, the 
test-level adjustment analysis as well as the change of the students’ performance levels after the 
adjustment constants were applied did not indicate strong evidence of device effects. 


The generalizability of the findings from this study may be limited due to the small sample size of both 
the paper students (for mode comparability) and the tablet students (for device comparability) at the 
high-school grades; however, it appears that high-quality matching supports the internal validity of this 
study’s findings. For mode and device comparability, there were little to no items flagged for mode or 
device effects, the psychometric properties of test scores were similar across assessment conditions, 
and any adjustments to student performance for the paper or tablet condition were within 
measurement precision. 


9.6 Evidence Based on Response Processes 


As noted in the AERA, APA, and NCME Standards (2014), additional support for a particular score 
interpretation or use can be provided by theoretical and empirical evidence indicating that test takers 
are using the intended response processes when responding to the items in a test. This type of evidence 
may be gathered from interacting with test takers in order to understand what processes underlie their 
item responses. Evidence may also be derived from feedback provided by test proctors/teachers 
involved in the administration of the test and raters involved in the scoring of constructed-response 
items. Evidence may also be gathered by evaluating the correct and incorrect responses to short 
constructed-response items (e.g., items requiring a few words to respond) or by evaluating the response 
patterns to multi-part items. 


PARCC has undertaken research investigating the quality of the items, tasks, and stimuli, focusing on 
whether students interact with items/tasks as intended, whether they were given enough time to 
complete the assessments, and the degree to which PARCC scoring rubrics allow accurate and reliable 
scoring. In addition, PARCC has examined the accessibility of the test for students with disabilities and 
English learners. This research has included examining students’ understanding of the format of the 
assessments and the use of technology. Although out of the purview of this technical report, several 
other PARCC research efforts have investigated questions relevant to response processes evidence.” 


° Various PARCC research is described at: http://parcc-assessment.org/research/ 
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9.7 Interpretations of Test Scores 


The PARCC ELA/L and mathematics scores are expressed as scale scores (both total scores and claim 
scores), along with performance levels to describe how well students met the academic standards for 
their grade level. Additionally, information on specific skills (the subclaims) is also provided and is 
reported as Below Expectations, Nearly Meets Expectations, and Meets or Exceeds Expectations. On the 
basis of a student’s total score, an inference is drawn about how much knowledge and skill in the 
content area the student has acquired. The total score is also used to classify students in terms of the 
level of knowledge and skill in the content area as students progress in the K-12 education. These levels 
are called performance levels and are reported as: 


e Level 5: Exceeded expectations 

e Level 4: Met expectations 

e Level 3: Approached expectations 

e Level 2: Partially met expectations 

e Level 1: Did not yet meet expectations 


Students classified as either Level 4 or Level 5 are meeting or exceeding the grade level expectations. 
PARCC has developed performance level descriptors (PLDs) to assist with the understanding and 
interpretations of the ELA/L and mathematics scores (http://parcc-assessment.org/performance-levels/ 
). Additionally, resource information is available online to educators, parents, and students 
(http://parcc-assessment.org/resources), which includes information on understanding and interpreting 
the ELA/L and mathematics score reports 
(https://www.isbe.net/Documents/Score_Report_Interpretation_Guide_Spring_2017.pdf 
http://understandthescore.org ). 


Section 12 of this technical report provides more information on the scale scores and the subclaim 
scores. 


9.8 Evidence Based on the Consequences to Testing 


The consequence of testing should also be investigated to support the validity evidence for the use of 
the PARCC assessments as the standards note that tests are usually administered “with the expectation 
that some benefit will be realized from the intended use of the scores” (AERA, APA, & NCME, 2014). 
When this is the case, evidence that the expected benefits accrue will provide support for the intended 
use of the scores. Evidence of the consequence of testing will also accrue with the continued 
implementation of the CCSS and the continued administration of the PARCC assessments. 


Consequences of the PARCC tests may vary by state or by school district. For example, some states may 
require “passing” the PARCC assessments as one of several criteria for high school graduation, while 
other states/districts may not require students to “pass” the PARCC assessments for high school 
graduation. Additionally, some school districts may use the PARCC scores along with other information 
such as school grades and teacher recommendations for placing students into special programs (e.g., 
remedial support, gifted and talented program) or for course placement (e.g., Algebra | in grade 8). 
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Because the consequences for the PARCC assessments can vary by each state, it is suggested that each 
PARCC member state provide school districts, teachers, parents, and students with information on how 
to interpret and use the PARCC scores. Additionally, the states should monitor how PARCC scores are 
used to ensure that the scores are being used as intended by PARCC. 


9.9 Summary 


In this section of the technical report, several aspects of validity were included, such as validity evidence 
based on content, the internal structure of the assessments, relationships across the content 
assessments, and evidence from special studies. 


The PARCC item development process involved educators, assessment experts, and bias and sensitivity 
experts in review of text, items, and tasks for accuracy, appropriateness, and freedom from bias. PARCC 
conducted several studies during the item development process to evaluate the item development 
process (e.g., technological functionalities, answer time required, and student experiences). 
Additionally, items were field tested prior to the initial operational administration, and data and 
feedback from students, test administrators, and classroom teachers was used to improve the 
operational administration of the items and to inform future item development. The multiple item and 
form reviews conducted by educators and studies to evaluate item administration help to ensure the 
integrity of the PARCC assessments. 


The intercorrelations of the subclaims, the reliability analyses, and the local item dependence analyses 
indicated that the ELA/L and the mathematics assessments are both essentially unidimensional. 
Furthermore, the correlations between ELA/L and mathematics indicated that the two assessments are 
measuring different content. Also, the patterns of correlations for the CBT and PBT assessments were 
similar, indicating that the structure of the assessments were similar across the two modes. 


Several studies were conducted as part of the PARCC assessment program (e.g., benchmarking study, 
content evaluation/alignment studies, longitudinal study, and mode and device comparability studies). 
The benchmarking study was conducted in support of the standard setting meeting. This study indicated 
students performing at or above Level 4 could be considered to be college- and career-ready or on track 
to readiness. 


The content evaluation/alignment studies performed by the Fordham Institute and HumRRO indicate 
that the PARCC assessments are good to excellent matches to the CCSS in terms of content and depth of 
knowledge. Thus, the PARCC assessments are assessing the college- and career-readiness standards. 
However, the reports noted that the PARCC program could improve by adding a wider range of depth of 
knowledge to some of the assessments. The reports also suggested enhancing the ELA/L assessments by 
including a research task that requires the use of two or more sources of information. 


In the longitudinal study of external validity, associations between PARCC performance levels and 
college-readiness benchmarks established by the College Board and ACT were used to study the claim 
that students who achieve Level 4 have a .75 probability of attaining at least a C in entry-level, credit- 
bearing, postsecondary coursework. In the first phase of the study, the relationship between PARCC and 
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external tests was studied. Overall, results indicated that a student meeting the benchmark on the 
PARCC test had a high probability of making the benchmark on the external test, but the converse did 
not hold for students meeting the benchmark on the external test, for the majority of comparisons. 
These results suggest that meeting the PARCC benchmark is an indicator of academic readiness for 
college. In the next phase of the study, the relationship between PARCC scores and performance in first- 
year college courses will be explored. 


The mode comparability study indicated that the comparability across modes was inconsistent across 
content domains and grade levels. The results of the mode comparability analysis were mixed and found 
to be consistent with prior research. The results of the device comparability study revealed consistent 
evidence supporting the comparability between the tablet condition (TC) and the non-tablet condition 
(NTC). In both the mode and device comparability studies, there were little to no items flagged for mode 
or device effects, the psychometric properties of test scores were similar across assessment conditions, 
and any adjustments to student performance for the paper or tablet condition were within 
measurement precision. 


In addition to the validity information presented in this section of the technical report, other 
information in support of the uses and interpretations of the PARCC scores appear in the following 
sections: 


Section 5 presents information regarding student characteristics for the spring administration of the 
ELA/L and mathematics administration. 


Section 6 provides information concerning the test characteristics based on classical test theory. 
Section 7 provides information regarding the differential item functioning (DIF) analyses. 


Section 8 provides information on the test reliability (total test score and for subclaims) and includes 
information on the interrater reliability/agreement. 


Section 12 provides detailed information concerning the scores that were reported and the cut scores 
for ELA/L and mathematics. 


The technical report addendum provides the test taker characteristics and test reliability (total test score 
and for subclaims) for the 2016 fall block administration. 
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Section 10: IRT Calibration and Scaling in Operational Year Three 


10.1 Overview 


Multiple operational core forms were administered for each grade in English language arts/literacy 
(ELA/L) and mathematics assessments. The purpose of the item response theory (IRT) calibration and 
scaling was to place all operational items for a single grade/subject onto a common scale. For the ELA/L 
computer-based tests (CBTs), the IRT parameters were post-equated. This section describes procedures 
used to calibrate and scale the post-equated PARCC operational assessments. Because ELA/L paper- 
based tests (PBTs) and all mathematics tests were pre-equated, much of the discussion in this section 
will not apply; however, the parameters used to construct the conversion tables for these tests are 
presented in this section. 


In this section of the technical report, the following topics related to IRT calibration and scaling are 
discussed: 


Calibration: 
10.2 IRT Data Preparation 
10.3 Description of the Calibration Process 
10.4 Model Fit Evaluation Criteria 
10.5 Items Excluded from Score Reporting 


Scaling: 
10.6 Scaling Parameter Estimates 
10.7 Items Excluded from Linking Sets 
10.8 Correlations and Plots of Scaling Item Parameter Estimates 
10.9 Scaling Constants 
10.10 Summary Statistics and Distributions from IRT Analyses 


10.2 IRT Data Preparation 
10.2.1 Overview 


Post-equating was performed on an early sample of the student data. The Bureau of Indian Education 
(BIE), the District of Columbia (DC), and six states participated in the spring administration: Colorado 
(CO), Illinois (IL), Maryland (MD), New Jersey (NJ), New Mexico (NM), and Rhode Island (RI). Each state 
tested over multiple weeks. Student data were monitored to determine when the early equating 
sampling criteria were met. Student data were evaluated for the following: 


1) Overall N-count, form count, and item count 
2) Demographic representation 

3) State representation 

4) Summative scale score distribution 
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Based on the early equating report, using data from the prior spring PARCC administration, 
approximately 25 percent (75,000—90,000) of the online student data were sufficient for post-equating 
selected tests in grades 3 through 8. Approximately 40 percent (70,000—90,000) of the online student 
data were sufficient for post-equating selected tests in grades 9 through 11. The larger percentage of 
the student data for high school assessments was due to the high percentage of students removed for 
not meeting attemptedness criteria and the need to obtain student responses for each score category 
for the more difficult items. The results from the research study were used to determine criteria for 
sample size and acceptable differences between the baseline demographic distributions and the sample 
demographic distributions for the post-equating sample. 


The resulting early equating samples for the spring administration exceeded state representation, 
exceeded the sample size criteria, met criteria for most of the demographic groups, and met criteria for 
the prior grade’s PARCC performance level distributions for most of the grades/subjects. Table 10.1 lists 
the equating sample sizes for all tests that were post-equated. 


All student response data in the early equating samples for operational items were used to create the 
IRT sparse data matrices for the concurrent calibration. IRT sparse data matrices combine student data 
across forms within administration mode. When duplicate records for a single student existed, the 
record with the largest raw score was included in the data file (and the other record was excluded). No 
student was included more than one time in the IRT sparse data matrices files. 


10.2.2 Student Inclusion/Exclusion Rules 


The following are the IRT valid case criteria. These criteria are the same as the student 
inclusion/exclusion rules used to evaluate and filter data prior to conducting the operational item 
analysis (IA) and differential item functioning (DIF) analyses (steps 1-5). The rules were agreed upon 
with PARCC and applied to the scored data used in the IRT calibration. 

1. All records with an invalid form number were excluded. 

All records that were flagged as “void” were excluded. 

3. Records in which the student attempted fewer than 25 percent of the items were excluded. An 
item was deemed “not attempted” if it had a value of “M” (item omitted) or “Z” (item “spoiled,” 
do not score) in the scored item response block. For example, if there were 25 items on a form 
and two were flawed (“Z”), those two items were not included in the numerator or denominator 
of the percentage-attempted calculation. 

4. For students with more than one valid record, the record with the higher raw score was chosen. 

5. Records for students with administration issues or anomalies were excluded. 


An assumption was made that mathematics items translated into Spanish were equivalent to the same 
items in English. Prior results of Soanish versus English DIF analyses supported this assumption. 


10.2.3 Items Excluded from IRT Sparse Matrices 


Pearson conducted an initial scoring and key check. Items identified by Pearson as “spoiled” (also 
referred to as “do not use (DNU)”) were listed and excluded from the test maps. When the IRT sparse 
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data matrices were created, all items were included in the files unless they were marked as “spoiled” by 
Pearson. 


10.2.4 Omitted, Not Reached, and Not Presented Items 


In the student data files, “Z’ was used to represent “spoiled” or “not presented” items and “M” was 
used to represent omitted items. For IA and IRT, omitted and not reached items were treated 
differently. Item response scores for omits were recoded as “O” in the IRT sparse matrix files (i.e., unless 
the omitted item was a “not reached” item). Not reached items are omitted items at the end of the test 
or unit—items that the student probably did not reach or try to answer. Not reached items were 
recoded from “M” in the SIRB to “N” (i.e., not presented) in the IRT sparse matrix files, if all items from 
that point to the end of the form are “M” or “Z.” Not reached items were counted as missing or no 
response, and therefore did not contribute to the item statistics for IA and IRT calibration. 


10.2.5 Quality Control of the IRT Sparse Matrix Data Files 


The IRT sparse data matrices were created by the primary analysts and replicators from Pearson and 
HumRRO. The matrices were checked for quality and accuracy by comparing the number of students (N- 
counts), item category frequencies, and item statistics (e.g., AIS values) between Pearson and HumRRO. 
Since the same inclusion rules for students were used, all N-counts, category frequencies, and statistics 
for all items matched. All discrepancies in N-counts were resolved. The programs used to create the IRT 
statistics were independent, so the QC procedure involved parallel computing. Table 10.1 shows the N- 
counts and number of items in the CBT IRT sparse data matrices for each grade in ELA/L. 


Table 10.1 N-Counts and Number of Items in the ELA/L IRT Calibration Files 


Grade N-Count Items 
3 319,709 82 
4 304,409 86 
5 333,896 85 
6 253,902 106 
7 230,200 98 
8 225,183 104 
9 177,216 96 
10 124,417 104 
11 95,487 106 


10.3 Description of the Calibration Process 


The IRT calibrations were performed only on the ELA/L CBT tests. The form-to-form linking is established 
through internal and external common items selected during test construction to represent the 
blueprint. 
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10.3.1 Two-Parameter Logistic/Generalized Partial Credit Model 


The operational IRT analyses were conducted by both Pearson and HumRRO. The operational items in 
the IRT sparse data matrix were concurrently calibrated with the two-parameter logistic/generalized 
partial credit model (2PL/GPC: Muraki, 1992). The 2PL/GPC is denoted 


vim(9}) = _ exp Xiao Da;(6; — bi + dix)| 
Leno XP [Leno Dai (6; — bi + dix) 
where a,(6; — bj + dig) = 03 Pim(9;) is the probability of a test taker with 6; getting score m on item i; 
D is the IRT scale constant (1.7); a; is the discrimination parameter of item i; b; is the item difficulty 
parameter of item i; dj, is the kt" step deviation value for item i; M, is the number of score categories of 
item i with possible item scores as consecutive integers from zero to M; — 1; v sequences through each 
response category through M; — 1. IRT calibrations might also use a guessing parameter in special cases, 
if needed. 


10.3.2 Treatment of Prose Constructed-Response (PCR) Tasks 


The prose constructed-response (PCR) tasks were calibrated at the trait score level (and not as 
aggregated scores). To address the issue of local independence related to PCR items, a single-calibration 
“model” approach was used. When sample sizes were large (i.e., greater than 10,000 test takers), the 
data were manipulated using random assignment, by selecting one of the two traits for each PCR item 
for each student. Then one calibration was run so that all trait parameters were independently 
estimated. When sample sizes were smaller (i.e., field-test samples), a multiple-calibration “model” 
approach was used. In this alternative approach, the same data set was calibrated two times, each trait 
represented in one of the two data sets for all students. Then the PCR traits were scaled onto the base 
scale using non-PCR items as anchor items. These two trait calibration approaches addressed the issue 
of local dependence while allowing for the accurate calculation of claim scores and the proper weighting 
of traits in the summative scale scores. 


10.3.3 IRT Item Exclusion Rules (Before Calibration) 


In addition to checking IRT data for accuracy, Pearson conducted item analyses (IA) to identify items that 
were not performing as expected and should be considered for removal from calibration and score 
reporting. The following are the criteria Pearson used to flag extremely problematic items to be dropped 
from calibration. All “non-spoiled” items were included in the IRT data matrices; however, the IRTPRO 
calibration software (Cai, Thissen, & du Toit, 2011) control files were used to exclude from calibration 
items flagged for the following reasons: 


1. A weighted polyserial correlation less than 0.0 

2. An average item score of 0.0 

3. 100 percent of the students having the same item score, such as: 
a. 100 percent omitted the item 
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100 percent received the same score 

100 percent of the responses were at the same score after collapsing score categories 
due to low frequencies, or 

100 percent of the responses were not presented or not reached 


Insufficient sample sizes for the selected IRT model combinations (i.e., 300 for the 2PL/GPC) 


5. High omit rates (i.e., greater than 50 percent) on one or more forms (usually an indication 


that an item may not be functioning correctly on all forms) 


A master list of all problematic items before and after calibration was maintained and all flagged and 


potentially flawed items were brought to the PARCC Priority Alert Task Force (consisting of Parcc Inc. 


and participating State Leads for PARCC member states) for content and statistical reviews. Ultimately, 


the decisions about whether to keep or exclude an item from score reporting was made by the PARCC 


Priority Task Force. 


10.3.4 IRTPRO Calibration Procedures and Convergence Criteria 


The data were calibrated concurrently across forms using the 2PL/GPC model combination. The primary 


goal was to place the operational item data within each content area and grade/subject on a common 


difficulty scale. The following are the steps used to calibrate the operational item response data: 


1. Using the IRT sparse data matrices, concurrent calibrations were conducted using 


commercially available IRTPRO for Windows (version 3.0) on CBT data, and separately on 


PBT data within each grade/subject. 
2. The 2PL/GPC model combination was used for all grades and subjects for each content area. 


Thus, two calibrations were completed for each grade/subject. 


3. IRTPRO Calibration Settings: The logistic partial credit model was specified using the scale 


constant of 1.0. The prior distributions for latent traits were set to a mean of zero and a 


standard deviation of one. The number of quadrature points used in the estimation was set 


to 49. And the slope starting value was set/updated before each run. 


4. Each IRTPRO run was inspected for convergence and for any unexpected item-parameter 


estimates. The PRIORS command in IRTPRO provided a prior on IRT parameters to constrain 


the calibration so that convergence was more likely. Specifically, option “Guessing[0]” 


indicated that the prior is placed on the lower asymptote for the 3-PL model, and a normal 


distribution for the priors with mean of -1.4 and standard deviation 1. For these items, an 


inspection of item-level statistics and modal-data fit plots were sufficient to ensure that 


item parameters were acceptable if convergence was reached. Item information functions 


from the IRTPRO output may also be reviewed. Pearson verified that the maximum number 
of EM (expectation-maximization) cycles was not reached (which indicated the program did 
not converge). 

To convert IRTPRO item parameters to the commonly used logistic parameter presentation 
(called new item parameters), the following formula was used since IRTPRO uses 1.0 fora 
scaling constant. There was no need to transfer b- and c-parameters from IRTPRO output. 
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Please note that all unscaled and scaled item parameters were kept on the theta scale. For 
2PL or 3PL models: 


Gs 
irtpro 


1.7 


New a-parameter: as = 


new 


Pearson reported any need for item-calibration decisions, including convergence issues and 
extreme parameter estimates, along with proposed resolutions, to the Priority Alert Task 
Force. Anticipated resolutions included fixing the slope parameters to a minimum .10 value, 
fixing the guessing parameter to a rational value (1 divided by number of options), and fixing 
the difficulty parameters at an upper or lower bound, depending on the nature of the 
problem. If extreme b-parameter values were observed (e.g., > 100) and the a-parameter 
values for these items were low (i.e., < 0.10), it was recommended that the prior for the a- 
parameter be set to 0.5. 


Dropping an item from further processing or dropping an item and rerunning IRTPRO was 
performed only if it was needed after communication with HumRRO and the Priority Alert 
Task Force. 


Inspection of model-data fit plots was helpful in deciding parameter constraints and 
acceptability of parameter fit. Documentation of each step, after resolution of any issues, 
was provided by Pearson to PARCC, HumRRO, and Measured Progress. 


10.3.5 Calibration Quality Control 


To ensure IRT calibrations and conversion tables were produced accurately, HumRRO replicated the IRT 


calibrations and the generation of the score conversion tables. Both Pearson and HumRRO used the 


same calibration software, IRTPRO. Daily meetings were held so that Pearson and HumRRO could 


provide status reports and discuss issues related to the IRT work. Measured Progress performed 


independent quality control comparisons between the Pearson and HumRRO item parameter estimates 


to identify any differences. 


Specifically, Measured Progress completed the following quality control analyses/comparisons: 


1. 


Made sure all items were treated the same way (e.g., if Pearson collapsed a category, made sure 
HumRRO collapsed the category in the same way for the item) 


Compared IRT item parameter estimates by Pearson and HumRRO (i.e., IRT a-, b-, and d- 
parameter estimates) 


Compared the scaling constants for the common item linking sets 


Compared transformed PBT parameter estimates generated by Pearson and HumRRO 
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5. Compared all conversion tables produced by Pearson and HumRRO to make sure they were 
accurate 


Measured Progress prepared reports documenting their findings. Exact matches were found between all 
Pearson and HumRRO conversion tables before scores were reported. 


10.4 Model Fit Evaluation Criteria 


The usefulness of IRT models is dependent on the extent to which they effectively reflect the data. As 
discussed by Hambleton, Swaminathan, and Rogers (1991), “The advantages of item response models 
can be obtained only when the fit between the model and the test data of interest is satisfactory. A 
poorly fitting IRT model will not yield invariant item and ability parameters” (p. 53). 


After convergence was achieved for each IRT data set, the IRT model fit was evaluated by doing the 
following: 

1. Calculating the Qi statistic and comparing it to a criterion score 

2. Calculating the G? statistic and comparing it to a criterion score 

3. Reviewing graphical output for all items 


The Qi statistic (Yen, 1981) was used as an index of correspondence between observed and expected 
performance. To compute Q,, first the estimated item parameters and student response data (along 
with observed item scores) were used to estimate student ability (@). Next, expected performance was 
computed on each item using students’ ability estimates in combination with estimated item 
parameters. Differences between expected item performance and observed item performance were 
then compared at 10 intervals across the range of student achievement (with approximately the same 
number of students per interval). Q: was computed as a ratio involving expected and observed item 
performance. Q: is interpretable as a chi-squared (7) statistic, which can be compared to a critical chi- 
squared value to make a statistical inference about whether the data (observed item performance) were 
consistent with what might be observed if the IRT model was true (expected item performance). Q:; is 
not directly comparable across different item types because items with different numbers of IRT 
parameters have different degrees of freedom (df). For that reason, a linear transformation (to a Z- 
score, Za,) was applied to Qi). This transformation also made item fit results easier to interpret and 


addressed the sensitivity of Qi to sample size. 


To evaluate item fit, Yen’s Qi statistic was calculated for all operational and field-test items. Q: is a fit 
statistic that compares observed and expected item performance. MAP (maximum a posteriori) 
estimates from IRTPRO were used as student ability estimates. For dichotomous items, Qi was 
computed as 
j 
2 
Nij (Oi; — Ej) 


O4= 
a Ei; — Eij) 


March 27, 2018 Page 177 


PARCC 2017 Technical Report 


where N;; was the number of examinees in interval (or group) j for item /, Oj was the observed 
proportion of the examinees for the same cell, and Fj; was the expected proportions of the examinees 
for the same interval. The expected proportion was computed as 


Nij 


=a) Pi(Ba) 


J ej 


where P; (6, ) was the item characteristic function for item i and examinee a. The summation is taken 
over examinees in interval j. 


The generalization of Q; for items with multiple response categories is 


10 Mj 


N;;(O; 
Geno > a ij ( a - Ex.j)? 
Ej j 


j=1k= 
where 


1 ON; 
Fixj = Ng uae Pix (6). 


Both Q; and generalized Q: results were transformed to ZQ: and were compared to a criterion ZQi crit to 
determine acceptable fit. The conversion formula was 


Qi — df 
ZQ, = == 
g2ar 
and 
_N_ 
ZQi, crit = 1500 * 4, 


where df is the degrees of freedom. The degrees of freedom is equal to the number of independent cells 
less the number of independent item parameters. For example, the degrees of freedom for polytomous 
items equals [10 x (number of score categories—1) — number of independent item parameters]. For the 
GPCM, the number of independent item parameters equals 1 (for the a parameter) plus the number of 
step values (e.g., for an item scored 0, 1, 2, 3: there are 3 independent step values—the b parameter is 
simply the mean of the step values and is not, therefore, independent). 


If Qi is found to be excessively sensitive (i.e., a large number of items are flagged for poor fit, even if 
their item fit plots look reasonable), a likelinood-ratio chi-squared statistic may be computed for each 
item (Muraki, 1997): 
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Wi m 


T, . 
GP =2 y > ho ia 
wi NuyiPin Ow) 


w=1h=1 


where Nip is the observed frequency of the h" categorical response to item i in interval w, N,,; is the 
number of examinees in interval w for item i, Pip (6,,) is the expected probability of observing the h* 
categorical response to item j for the mean @ in interval w, and W; is the number of intervals remaining 
after neighboring intervals are merged, if necessary, to avoid expected values, N,,; Pin (Oy), less than 5. 
To conduct a standard hypothesis test, the number of degrees of freedom is equal to the number of 
intervals, W;, multiplied by m; — 1. 


As an alternative to a traditional hypothesis test, the “contingency coefficient” (effect size; Barton & 
Huynh, 2003) was computed: 


In this formula, G? was substituted for v7, and N is the sample size on which the IRT parameters were 
estimated. According to Cohen (1988, pp. 224-225), values of C below .10 are considered insignificant, 
.10+ small, .287+ medium, and .447+ large. A threshold of .35 is recommended (i.e., flag items for which 
C 2.35). 


An item fit-plot was created for each item. Item-fit plots show observed and expected average scores for 
each interval (e.g., figure below). 
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Figure 10.1 An example ELA/L 5-Category Item, 2 PL/GPC Model, N-count 44,658, Q1=1266.64, 
ZQ1=147.21 and a criterion ZQ1,crit= 237.02 


10.5 Items Excluded from Score Reporting 


As mentioned previously, after calibration and model fit evaluation were completed, a master list of all 
problematic items was compiled and potentially flawed items were brought to the PARCC Priority Alert 
Task Force. The Task Force reviewed each item, its content and the statistical properties, and made 
decisions about whether to include the item in the operational scores. Sometimes, an item was rejected 
because it appeared to have content issues, and sometimes an item was excluded because it could not 
be calibrated or showed extremely poor IRT model fit. Ultimately the decisions about whether to keep 
or exclude each flagged item was made by the Task Force. 


10.5.1 Item Review Process 


The following are the types of problematic items that were brought to the PARCC Priority Alert Task 
Force for evaluation and an “include or exclude” determination was made: 


1. Extremely difficult items (e.g., an item with a p value less than 0.02) 


2. Items with low a-parameter estimates (e.g., slope less than 0.10) 
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3. Items flagged for subgroup DIF or language DIF 


Again, the primary goal was to minimize the number of items dropped from the operational test forms. 
An equally important goal was to not advantage or disadvantage any test takers. 


10.5.2 Count and Percentage of Items Excluded from Score Reporting 


All items that did not have IRT item parameter estimates were excluded from the student operational 
scores and the conversion tables used for score reporting. For ELA/L and mathematics, at most 2 
percent of the items were excluded from score reporting for all grades/subjects. Figure 10.2 
demonstrates an item that was flagged during the calibrations and item fit review for poor statistics (a- 
parameter=0.02 and b-parameter=33.48) and poor fit (e.g. Qi= 4947.19, ZQi= 845.52 and a criterion 
ZQicrit= 468.90). This item was reviewed by the Priority Alert Task Force and removed from scoring. 


16:43 Friday, May 27,2016 4 
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Figure 10.2 An example ELA/L 3-Category Item, 2 PL/GPC Model, N-count 175,839 


Table 10.2 presents the count and percentage of CBT items excluded from IRT calibration along with the 
reasons the items were excluded. 
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Table 10.2 Number and Percentage of ELA/L Items Excluded from IRT Calibration 
Reason Excluded 


n of CBT 
Total n of Percent Small ¥ 
Grade Items Poor IA Did Not 
CBT Items Excluded Sample : Other 
Excluded . Stats Calibrate 
Size 
3 82 0 0% 
4 86 0 0% 
5 86 1 1% Display malfunction 
6 106 0 0% 
7 98 0 0% 
8 104 0 0% 
9 96 0 0% 
10 104 0 0% 
11 106 0 0% 


10.6 Scaling Parameter Estimates 


Year-to-year linking was performed on all ELA/L CBTs to transform IRT parameters to the 2016 IRT scale. 
The linking analyses included common-item sets. The linking methodology was based on the Stocking 
and Lord (1983) test characteristic curve scale transformation method. 


Year-to-year linking transforms IRT parameters from different years (or administrations) onto the same 
underlying IRT scale. Due to the PARCC test redesign and updates to the trait scoring for the PCRs, the 
spring 2016 online IRT scale was established as the base IRT scale. 


The entire item bank was previously transformed to the spring 2016 IRT scale through a common-item 
linking set consisting of spring 2016 operational online items that were operational or field-test items on 
the spring 2015 online administration. The details for this procedure can be found in the 2016 PARCC 
Technical Report. 


HumRRO not only conducted independent calibrations of item response data using IRTPRO scaling 
software, they also used STUIRT (Kim & Kolen, 2004) software to transform their IRTPRO item parameter 
estimates onto the IRTPRO scales for each grade/subject. HumRRO’s scaling constants were compared 
to those generated by Pearson and found exactly match. As described in Section 10.3.4, Measured 
Progress independently made certain that the same items were excluded from the linking sets, and 
compared transformed parameter estimates by Pearson and HumRRO. 


10.7 Items Excluded from Linking Sets 


Linking is an iterative process. Robust Z (Huynh & Meyer, 2010) and Weighted Root Mean Square 
Difference (WRMSD) were used to identify outlier items in the linking sets. Furthermore, items used in 
the paper-to-online linking were checked for differential item functioning (DIF). Mantel-Haenszel D-DIF 


March 27, 2018 Page 182 


PARCC 2017 Technical Report 


procedures were used for dichotomous items and standardized mean difference (SMD) were calculated 
for polytomous items. The following rules were used to identify items for possible exclusion from the 
linking sets: 


1) Exclude an item from the common-item set if different amounts of collapsing resulted ina 
different number of response categories across modes or versions. 

2) Flag and potentially exclude an item from the common-item set if the online or paper weighted 
polyserial correlation, based on the item analysis, was less than 0.10. 

3) Exclude items dropped by the PARCC Priority Alert Task Force (i.e., due to content or parameter 
estimation issues). 

4) Flag and potentially exclude an item from the common-item set if the mode DIF results 
indicated positive or negative C-DIF. 


After removing items due to item performance issues as described above, the following steps were 
performed: 


5) Implement the Robust Z approach to see if any common items are flagged. 

6) Run the initial Stocking and Lord procedure using the STUIRT software. 

7) Calculate WRMSD and check to see if any common items exceed the threshold. 
8) Re-run STUIRT after removing the items flagged by Robust Z. 

9) Compare the slopes and intercepts from steps 2 and 4. 


SAS code was developed to calculate WRMSD, Robust Z, to compare the item characteristic curves (ICCs) 
across modes, and to identify items for possible removal from the linking sets. Table 10.3 lists the flag 
criteria for the WRMSD. (Note: these values were originally developed as part of the 2014 PARCC field- 
test analysis.). 


Table 10.3 WRMSD Flagging Criteria for Inspection and Possible Removal of Linking Items 


WRMSD/ 
Categories Points Points WRMSD 
2 1 0.100 0.100 
3 2 0.075 0.150 
4 3 0.075 0.225 
5 4 0.075 0.300 
6 5 0.075 0.375 
7 6 0.075 0.450 
>=8 >= 7 0.090 0.999 


When inspecting items flagged for exclusion from the linking sets, content representation was also 
considered to avoid removing large numbers of items from the same subclaim. Table 10.4 presents the 
total number of common items, items excluded from the year-to-year linking sets, and items kept in the 
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linking sets for each grade for ELA/L. The final number of linking items ranged from 6 (in grade 3) to 22 
(in grade 9). Grades 6 and 7 had the largest number of items removed from the linking sets due to 
Robust Z for the a-parameter and b-parameter. 


Table 10.4 Number of ELA/L Items Excluded from the Year-to-Year Linking Sets 


Total n Final Number of Excluded Items by Reason for 
Giada of Number Number Exclusion 
Common Excluded _ in Linking Low Robust Z Robust Z High 
Items Set Polyserial IRT_a IRT_b WRMSD 

3 8 2 6 0 1 1 0 
4 15 1 14 0 0 1 0 
5 15 1 14 0 0 1 0 
6 13 3 10 0 2 1 0 
7 18 3 15 0 2 di 0 
8 14 2 12 0 0 2 0 
9 24 Z 22 0 2 0 0 
10 13 2 11 0 0 2 0 
11 8 0 8 0 0 0 0 


10.8 Correlations and Plots of Scaling Item Parameter Estimates 


Once the final group of items for each linking set was determined, the a- and b-parameter estimates 
were plotted and the correlation between the a-parameter estimates and the b-parameter estimates 
were calculated. Table 10.5 presents the number of linking items, score points of the linking items, and 
the correlation of the a- and b-parameter estimates across years. 


Table 10.5 Number of Items, Number of Points, and Correlations for ELA/L Year-to-Year Linking Items 


Number Parameter Correlations 
Grade Items Points a- b- 

3 6 12 0.9984 0.9998 
4 14 31 0.9815 0.9991 
5 14 30 0.9965 0.9027 
6 10 25 0.9994 0.9553 
7 15 33 0.9921 0.9982 
8 12 29 0.9973 0.9978 
9 22 47 0.9796 0.9821 
10 11 22 0.9806 0.9890 
11 8 19 0.9974 0.9985 
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Figures 10.3 to 10.4 are a selection of plots of the a- and b-parameter estimates for linking items for the 
year-to-year linking. ELA/L grade 8 (Figures 10.3 and 10.4) is an example of the year-to-year linking. For 
each plot, the x-axis is the original (reference) parameter and the y-axis is the new parameter after 
applying the scaling constants. 
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Figure 10.3 ELA/L Grade 8 Transformed New a- vs. Reference a-Parameter Estimates for Year-to-Year 
Linking 


March 27, 2018 Page 185 


PARCC 2017 Technical Report 


irt_b 2017 
oO 


irt_b 2016 


Figure 10.4 ELA/L Grade 8 Transformed New b- vs. Reference b-Parameter Estimates for Year-to-Year 
Linking 


10.9 Scaling Constants 


Table 10.6 presents the slope and intercept scaling constants for ELA/L for the year-to-year linking, 
derived from STUIRT (Kim & Kolen, 2004) using the Stocking and Lord (1983) test characteristic curve 
procedure. The slopes and intercepts are similar. The slopes range from .9929 to 1.0324, and the 
intercepts range from -0.0631 and 0.1076. 
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Table 10.6 Scaling Constants Spring 2016 to Spring 2017 for ELA/L 
Spring 2016 to Spring 2017 


Grade/Subject Slope Intercept 
3 1.0281 0.0926 
4 1.0324 0.0338 
5 1.063 0.1076 
6 0.9929 0.0608 
7 0.9959 -0.0406 
8 1.0218 -0.0631 
9 1.0306 0.0815 
10 0.9975 0.06 
11 1.0298 -0.0199 


10.10 Summary Statistics and Distributions from IRT Analyses 


Tables 10.7 through 10.15 present summary statistics for the IRT (b- and a-) parameter estimates, the 
standard errors (SEs) of the parameter estimates, and the IRT model fit values (chi-square and adjusted 
fit) for ELA/L CBTs. For all other tests, which were pre-equated, the summary statistics for the parameter 
estimates and the administration from which the parameter was first obtained are presented. The 
information is provided by content area (ELA/L and mathematics) and by mode (CBT and PBT) for all 
items at each grade level or course. The summary statistics shown include the total number of items and 
score points, along with the mean, standard deviation (SD), minimum, and maximum. 


10.10.1 IRT Summary Statistics for English Language Arts/Literacy 


Table 10.7 shows the b- and a-parameter estimates for the ELA/L CBT assessments. Table 10.8 presents 
the standard errors (SE) of the estimates, and Table 10.9 provides model fit information for the ELA/L 
CBT assessments. IRT summary statistics are provided in the Appendix 10 for ELA/L for all items, reading- 
only, and writing-only. Table 10.10 shows the summary of the parameters estimates, and Table 10.11 
shows the source year for the item statistics for each of the ELA/L PBT assessments. 
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Table 10.7 CBT IRT Summary Parameter Estimates for All Items for ELA/L by Grade 


Summary of b Estimates Summary of a Estimates 

No. of 

No.of Score 
Grade Items Points Mean SD Min Max Mean sD Min Max 
3 82 180 0.47 1.05 -1.4 3.44 0.55 0.2 0.11 1.00 
4 86 193 0.20 1.43 -6.48 4.99 0.49 0.22 0.13 0.97 
5 85 192 0.65 1.21 -6.27 3.56 0.47 0.23 0.13 1.00 
6 106 233 0.53 0.87 -0.97 4.45 0.48 0.19 0.15 0.96 
7 98 217 0.35 0.89 -2.17 3.82 0.48 0.23 0.14 1.11 
8 104 232 0.27 0.95 -2.39 3.30 0.46 0.26 0.09 1.10 
9 96 210 0.58 0.83 -1.04 3.59 0.47 0.28 0.10 1.40 


10 104 232 0.69 0.80 -1.27 3.06 0.48 0.23 0.16 1.09 
11 106 236 0.91 0.82 -0.54 3.77 0.48 0.19 0.14 0.91 


Table 10.8 CBT IRT Standard Errors of Parameter Estimates for All Items for ELA/L by Grade 


SE of b Estimates SE of a Estimates 

No. of 

No.of Score 
Grade Items’ Points Mean SD Min Max Mean SD Min Max 
3 82 180 0.009 0.007 0.004 0.053 0.004 0.002 0.002 0.01 
4 86 193 0.013 0.022 0.004 0.163 0.004 0.002 0.002 0.01 
5 85 192 0.012 0.017 0.004 0.14 0.004 0.002 0.002 0.01 
6 106 233 0.01 0.01 0.004 0.093 0.004 0.002 0.002 0.01 
7 98 217 0.01 0.008 0.004 0.067 0.005 0.002 0.002 0.012 
8 104 232 0.012 0.01 0.004 0.075 0.005 0.003 0.002 0.013 
9 96 210 0.012 0.011 0.004 0.067 0.005 0.003 0.002 0.017 


10 104 232 0.015 0.01 0.005 0.08 0.006 0.003 0.003 0.016 
11 106 236 0.019 0.015 0.007 0.099 0.007 0.003 0.003 0.016 
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GC Qu 
No. of 
No.of Score 
Grade Items Points Mean sD Min Max Mean SD Min Max 
3 82 180 2628.5 2558.7 182.3 116765 2524.8 2468.9 178.9 10234.5 
4 86 193 2566 2756.7 391.7 20517.1 2488.4 2633.2 373.6 17734.4 
5 85 192 3010.8 3148.6 132.3 20149.4 9706.6 62643.1 129.6 579715.1 
6 106 233 2033.8 1846.4 227.5 12312.5 1964.5 1828.4 222.5 12034 
7 98 217 2007 2081.5 189.3 11308.1 1979.5 2096.7 180.7 10699.5 
8 104 232 1922.6 1726.3 164.2 9222.1 1853.3 1678.6 152.6 8390.7 
9 96 210 1683.8 1348.7 150.2 5455.1 1671.9 1443.7 148.7 7125.8 
10 104 232 1154.7. 1286 58.1 11355.6 1127.9 1500.8 57.2 14014.8 
11 106 236 747.9 700.2 74.9 5082.1 700.3 665 75.6 4584.2 


Table 10.10 PBT IRT Summary Parameter Estimates for All Items for ELA/L by Grade 


Summary of b Estimates 


Summary of a Estimates 


No. of 

No.of Score 
Grade Items’ Points Mean SD Min Max Mean SD Min Max 
3 32 70 0.82 0.96 -0.9 3.35 0.5 0.18 0.12 0.82 
4 34 76 0.51 1.09 -1.08 4.96 0.48 0.24 0.17 0.99 
5 34 76 1.03 0.83 -0.97 2.99 0.43 0.22 0.13 0.94 
6 40 89 0.51 0.78 -0.74 2.37 0.44 0.19 0.15 0.96 
7 40 89 0.38 0.8 -1.38 2.15 0.38 0.16 0.19 0.84 
8 40 89 0.35 0.78 -1.39 1.9 0.43 0.19 0.1 0.84 
9 40 89 0.58 0.77 -1.07 2.22 0.44 0.21 0.18 0.96 
10 40 89 0.48 0.73 -1.05 1.88 0.51 0.19 0.18 0.9 
11 40 89 1.35 1 -0.58 3.75 0.41 0.21 0.15 0.9 
March 27, 2018 Page 189 


PARCC 2017 Technical Report 


Table 10.11 PBT IRT Parameter Distribution by Year for All Items for ELA/L by Grade 


Grade ALL 2014 2015 2016 
3 32 0 10 22 
4 34 0 8 26 
5 34 0 6 28 
6 40 0 12 28 
7 40 0 14 26 
8 40 0 0 40 
9 40 0 11 29 
10 40 0 22 18 
11 40 0 8 32 


10.10.2 IRT Summary Statistics for Mathematics 


Tables 10.12 and 10.14 show the b- and a-parameter estimates for the CBT and PBT mathematics 
assessments, respectively. Tables 10.13 and 10.15 show the source year for the item statistics for each 
of the assessments. IRT summary statistics are provided in the Appendix 10 for mathematics for all 
items, single-select multiple-choice items, constructed-response items, and subclaims. 


Table 10.12 CBT IRT Summary Parameter Estimates for All Items for Mathematics by Grade/Subject 


Summary of b Estimates Summary of a Estimates 
No. of 
No.of Score 
Grade Items Points Mean SD Min Max Mean SD Min Max 
3 113 172 -0.01 1.13 -3 2.81 0.77 0.25 0.22 1.42 
4 99 155 0.13 0.93 -1.98 2.1 0.76 0.19 0.29 1.23 
5 103 164 0.22 1.08 -2.44 2.61 0.68 0.25 0.22 1.45 
6 95 163 0.28 1.1 -2.98 2.46 0.72 0.25 0.26 1.38 
7 97 163 1.07 1.08 -1.76 3.47 0.7 0.28 0.24 1.71 
8 91 166 1.1 1.07 -1.8 3.92 0.65 0.26 0.23 1.35 


Al 104 191 1.53 0.92 -0.73 5.3 0.67 0.28 0.23 1.34 
GO 110 206 1.14 0.98 -1.77 3.88 0.77 0.33 0.16 1.86 
A2 102 194 1.56 0.91 -0.51 4.21 0.7 0.33 0.18 1.56 


M1 42 81 1.15 1.07 -0.95 3.94 0.64 0.22 0.18 1.28 
M2 41 80 1.9 1.45 -0.74 5.99 0.53 0.27 0.12 1.18 
M3 40 81 1.27 1.08 -2.27 4.28 0.59 0.27 0.16 1.27 


Note: Ai = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics II, M3 = Integrated Mathematics III. 
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Table 10.13 CBT IRT Parameter Distribution by Year for All Items for Mathematics by Grade/Subject 


Grade ALL 2014 2015 2016 
3 113 1 60 52 
4 99 2 58 39 
5 103 0 55 48 
6 95 0 46 49 
7 97 1 46 50 
8 91 0 43 48 

Al 104 0 45 59 
A2 102 0 35 67 
GO 110 0 38 72 
M1 42 0 20 22 
M2 Al 0 0 Al 
M3 40 0 0 40 


Table 10.14 PBT IRT Summary Parameter Estimates for All Items for Mathematics by Grade/Subject 


Summary of b Estimates Summary of a Estimates 

No. of 

No.of Score 
Grade Items’ Points Mean SD Min Max Mean sD Min Max 
3 43 66 -0.27 1.04 -2.44 2.08 0.74 0.26 0.29 1.42 
4 40 66 -0.18 1.12 -2.11 2.1 0.75 0.23 0.42 1.28 
5 40 66 0 1.16 -2.27 2.28 0.71 0.23 0.4 1.36 
6 38 66 0.27 1.32 -3.54 3.59 0.64 0.24 0.23 1.4 
7 38 66 0.93 1.2 -1.66 3.06 0.61 0.21 0.34 1.06 
8 36 66 1.04 1.17 -1.17 3.22 0.56 0.24 0.18 1.23 
Al 42 81 1.05 1.14 -1.16 3.27 0.54 0.2 0.17 1.01 
GO 43 81 1.02 1.1 -1.1 3.6 0.71 0.32 0.24 1.41 
A2 41 81 1.75 1.87 -1.39 9.06 0.61 0.29 0.12 1.41 
M1 42 81 1.22 1.31 -1.25 5.41 0.59 0.28 0 1.36 
M2 42 80 1.67 1.14 -0.5 4.87 0.52 0.24 0.18 1.15 
M3 39 80 1.31 1.33 -2.27 4.61 0.56 0.26 0.13 1.24 


Note: Ai = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics I, M2 = Integrated 
Mathematics II, M3 = Integrated Mathematics III. 
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Table 10.15 PBT IRT Parameter Distribution by Year for All Items for Mathematics by Grade/Subject 


Grade ALL 2014 2015 2016 
3 43 0 6 37 
4 40 0 9 31 
5 40 0 8 32 
6 38 0 11 27 
7 38 0 11 27 
8 36 0 5 31 
Al 42 0 10 32 
A2 41 0 2 39 

GO 43 0 10 33 
M1 42 1 19 22 
M2 42 0 22 20 
M3 39 0 14 25 
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Section 11: Performance Level Setting 


11.1 Performance Standards 


Performance standards relate levels of performance on an assessment directly to what students are 
expected to learn. This is done by establishing threshold scores that distinguish between performance 
levels. Performance level setting (PLS) is the process of establishing these threshold scores that define 
the performance levels for an assessment. 


11.2 Performance Levels and Policy Definitions 
For the PARCC assessments, the performance levels are 


e Level 5: Exceeded expectations 

e Level 4: Met expectations 

e Level 3: Approached expectations 

e Level 2: Partially met expectations 

e Level 1: Did not yet meet expectations 


More detailed descriptions of each performance level, known as policy definitions, are: 


Level 5: Exceeded expectations 
Students performing at this level exceed academic expectations for the knowledge, skills, and practices 
contained in the standards assessed at their grade level or course. 


Grades 3-10: Students performing at this level exceed academic expectations for the knowledge, skills, 
and practices contained in the standards for English language arts/literacy (ELA/L) or mathematics 
assessed at their grade level. They are academically well prepared to engage successfully in further 
studies in this content area. 


Algebra Il, Integrated Mathematics Ill, and ELA/L Grade 11: Students performing at this level exceed 


academic expectations for the knowledge, skills, and practices contained in the mathematics and ELA/L 
standards assessed at grade 11. They are very likely to engage successfully in entry-level, credit-bearing 
courses in mathematics and ELA/L, as well as technical courses requiring an equivalent command of the 
content area. Students performing at this level are exempt from having to take and pass placement tests 
in two- and four-year public institutions of higher education designed to determine whether they are 
academically prepared for such courses without need for remediation. 


Level 4: Met expectations 
Students performing at this level meet academic expectations for the knowledge, skills, and practices 
contained in the standards assessed at their grade level or course. 


Grades 3-10: Students performing at this level meet academic expectations for the knowledge, skills, 
and practices contained in the standards for ELA/L or mathematics assessed at their grade level. They 
are academically prepared to engage successfully in further studies in this content area. 
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Algebra Il, Integrated Mathematics Ill, and ELA/L Grade 11: Students performing at this level meet 
academic expectations for the knowledge, skills, and practices contained in mathematics and ELA/L at 
grade 11. They are very likely to engage successfully in entry-level, credit-bearing courses in 
mathematics and ELA/L, as well as technical courses requiring an equivalent command of the content 
area. Students performing at this level are exempt from having to take and pass placement tests in two- 
and four-year public institutions of higher education designed to determine whether they are 
academically prepared for such courses without need for remediation. 


Level 3: Approached expectations 
Students performing at this level approach academic expectations for the knowledge, skills, and 
practices contained in the standards assessed at their grade level or course. 


Grades 3-10: Students performing at this level approach academic expectations for the knowledge, 
skills, and practices contained in the standards for ELA/L or mathematics assessed at their grade level. 
They are likely prepared to engage successfully in further studies in this content area. 


Algebra Il, Integrated Mathematics III, and ELA/L Grade 11: Students performing at this level approach 
academic expectations for the knowledge, skills, and practices contained in the ELA/L and mathematics 
standards assessed at grade 11. They are likely to engage successfully in entry-level, credit-bearing 
courses in mathematics and ELA/L, as well as technical courses requiring an equivalent command of the 
content area. Students performing at Level 3 are strongly encouraged to continue to take challenging 
high school coursework in English and mathematics through graduation. Postsecondary institutions are 
encouraged to use additional information about students performing at Level 3, such as course 
completion, course grades, and scores on other assessments to determine whether to place them 
directly into entry-level courses. 


Level 2: Partially met expectations 
Students performing at this level partially meet academic expectations for the knowledge, skills, and 
practices contained in the standards assessed at their grade level or course. 


Grades 3-10: Students performing at this level partially meet academic expectations for the 
knowledge, skills, and practices contained in the standards for ELA/L or mathematics assessed at their 
grade level. They will likely need academic support to engage successfully in further studies in this 
content area. 


Algebra Il, Integrated Mathematics Ill, and ELA/L Grade 11: Students performing at this level partially 
meet academic expectations for the knowledge, skills, and practices contained in the ELA/L and 
mathematics standards assessed at grade 11. They will likely need academic support to engage 
successfully in entry-level, credit-bearing courses, and technical courses requiring an equivalent 
command of the content area. Students performing at this level are not exempt from having to take and 
pass placement tests designed to determine whether they are academically prepared for such courses 
without the need for remediation in two- and four-year public institutions of higher education. 
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Level 1: Did not yet meet expectations 
Students performing at this level do not yet meet academic expectations for the knowledge, skills, and 
practices contained in the standards assessed at their grade level or course. 


Grades 3-10: Students performing at this level do not yet meet academic expectations for the 
knowledge, skills, and practices contained in the standards for ELA/L or mathematics assessed at their 
grade level. They will need academic support to engage successfully in further studies in this content 
area. 


Algebra Il, Integrated Mathematics III, and ELA/L Grade 11: Students performing at this level do not yet 


meet academic expectations for the knowledge, skills, and practices contained in the ELA/L and 
mathematics standards assessed at grade 11. They will need academic support to engage successfully in 
entry-level, credit-bearing courses in college algebra, introductory college statistics, and technical 
courses requiring an equivalent level of mathematics. Students performing at this level are not exempt 
from having to take and pass placement tests in two- and four-year public institutions of higher 
education designed to determine whether they are academically prepared for such courses without 
need for remediation. 


11.3 Performance Level Setting Process for the PARCC Assessment System 


One of the main objectives of the PARCC assessment system is to provide information to students, 
parents, educators, and administrators as to whether students are on track in their learning for success 
after high school, defined as college- and career-readiness. To set performance levels associated with 
this objective, PARCC used the evidence-based standard setting (EBSS) method (Beimers, Way, McClarty, 
& Miles, 2012) for the PARCC PLS process. The EBSS method is a systematic method for combining 
various considerations into the process for setting performance levels, including policy considerations, 
content standards, educator judgment about what students should know and be able to demonstrate, 
and research to support PARCC’s policy goals related to college- and career-readiness. A defined 
multistep process was used to allow a diverse set of stakeholders to consider the interaction of these 
elements in recommending performance level threshold scores for each PARCC assessment. 


The seven steps of the EBSS process that were followed in order to establish performance standards for 
the PARCC assessments are: 


Step 1: Define outcomes of interest and policy goals 

Step 2: Develop research, data collection, and analysis plans 

Step 3: Synthesize the research results 

Step 4: Conduct pre-policy meeting 

Step 5: Conduct performance level setting (PLS) meetings with panels 
Step 6: Conduct reasonableness review with post-policy panel 

Step 7: Continue to gather evidence in support of standards 


A summary of key components within these steps is provided below. Additional detail about each step 
in the PARCC PLS process is provided in the PARCC Performance Level Setting Technical Report. 
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11.3.1 PARCC Research Studies 


PARCC conducted two research studies in support of their policy goals—the benchmarking study and the 
postsecondary educators’ judgment (PEJ) study. The benchmarking study included a review of the 
literature relative to college- and career-readiness as well as consideration of the percentage of students 
obtaining a level equivalent to college- and career-readiness on a set of external assessments (e.g., ACT, 
SAT, NAEP). The PEJ study involved a group of nearly 200 college faculty reviewing items on the Algebra 
Il and ELA/L grade 11 PARCC assessments and making judgments about the level of performance needed 
on each item to be academically ready for an entry-level college-credit bearing course in mathematics or 
ELA/L. Additional detail?® about the benchmarking study can be found in the PARCC Performance Level 
Setting Technical Report as well as in the PARCC Benchmarking Study Report. Additional detail about the 
PEJ study can be found in the PARCC Performance Level Setting Technical Report as well as in the 
Postsecondary Educators’ Judgment Study Final Report. 


11.3.2 PARCC Pre-Policy Meeting 


Prior to the PLS meetings, a pre-policy meeting was convened to determine reasonable ranges that 
would be shown to panelists during the high school PLS meetings. Pre-policy meeting participants 
included representatives from both K-12 and higher education who served in roles such as 
commissioner/superintendent, deputy/assistant commissioner, state board member, director of 
assessment, director of academic affairs, senior policy associate, and so on. The reasonable ranges 
recommended by the pre-policy meeting defined the minimum and maximum percentage of students 
that would be expected to be classified as college- and career-ready. The pre-policy meeting participants 
reviewed the test purpose, how the performance standards will be used, and the results of the research 
studies to provide the recommendations for the reasonable ranges without viewing any student 
performance data. 


11.3.3 Performance Level Setting Meetings 


The task of the PLS committee was to recommend four threshold scores that would define the five 
performance levels for each PARCC assessment. PARCC solicited nominations from all states that had 
administered the PARCC assessments in 2014—2015 for panelists to serve on the PLS committees. 
Nominations were solicited both from state departments of public education (K-12) and higher 
education (primarily for participation on the high school panels). When selecting panelists, PARCC 
placed an emphasis on those educators who had content knowledge as well as experience with a variety 
of student groups and attempted to balance the panels in terms of state representation. 


PARCC used an extended modified Angoff (Yes/No) method to collect educator judgments on the PARCC 
items. This method asked panelists to review each item on a reference form of the PARCC assessment 
and to make the following judgment: 


10 More information can be requested online from http://www.parcconline.org/assessments/test-design/research. 
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How many points would a borderline student at each performance level likely earn if they answered 
the question? 


This extension to the Yes/No standard setting method (Plake, Ferdous, Impara, & Buckendahl, 2005) 
allowed for incorporation of the multipoint PARCC items by asking educators to evaluate (Yes or No) 
whether a borderline student would earn the maximum number of points on an item, a lesser number 
of points on an item, or no points on the item. In the case of a single point or multiple-choice item, this 
task simplifies to the standard Yes/No method. 


After receiving training on the PLS procedure, panelists participated in three rounds of judgments for 
each assessment. Within each round, panelists were asked to consider the items in the test form, 
starting with the performance-based assessment (PBA) component and then the end-of-year (EOY) 
component. Each panelist made a judgment for the Level 2 performance level, followed by judgments 
for the Level 3 performance level, the Level 4 performance level, and the Level 5 performance level, in 
this order. The panelists entered their item judgments for each round by completing an online item 
judgment survey. Educator judgments were summed across items to create an estimated total score on 
the reference form for each performance level threshold. Feedback data relative to panelist agreement, 
student performance on the items, and student performance on the test as a whole were provided in 
between each of the three rounds of judgment. Panelists were shown the pre-policy reasonable ranges 
prior to making their Round 1 judgments and again as feedback data following each round of judgment. 


A dry-run of the PARCC PLS meeting process was held for grade 11 ELA/L and Algebra Il in order to 
evaluate the implementation of the PLS method with the innovative characteristics of the PARCC 
assessments. These content areas were selected because they combined all the various aspects of the 
PARCC assessments, including the various types of items, scoring rules, and performance level decisions. 
The dry-run PLS meetings provided the opportunity to implement and evaluate multiple aspects of the 
operational plan for the actual PLS meeting, including pre-work, meeting materials, data analysis and 
feedback, and staff and panelist functions. The results of the dry-run PLS meeting were used to 
implement improvements in the process for the operational PLS meetings. Additional information about 
the methods and results of the dry-run PLS meeting is available in the full report in the PARCC 
Performance Level Setting Dry-Run Meeting Report. 


The PLS meetings for the PARCC assessments were conducted during three one-week sessions. The 
dates of the twelve PLS committee meetings that were conducted are shown in Table 11.1. 


March 27, 2018 Page 197 


PARCC 2017 Technical Report 


Table 11.1 PARCC PLS Committee Meetings and Dates 


Dates Committees by Subjects and Grades 


July 27-31, 2015 Algebra |/Integrated Mathematics | 


Geometry/Integrated Mathematics I| 


Algebra II/Integrated Mathematics III 


Grade 9 English Language Arts/Literacy 


Grade 10 English Language Arts/Literacy 


Grade 11 English Language Arts/Literacy 


August 17-21, 2015 Grades 7 & 8 Mathematics 
Grades 7 & 8 English Language Arts/Literacy 
August 24-28, 2015 Grades 3 & 4 Mathematics 


Grades 5 & 6 Mathematics 


Grades 3 & 4 English Language Arts/Literacy 


Grades 5 & 6 English Language Arts/Literacy 


Additional information about the methods and results of the PLS meetings is available in the 
Performance Level Setting Technical Report. 


11.3.4 PARCC Post-Policy Reasonableness Review 


Performance standards for all PARCC assessments were recommended by PLS committees and reviewed 
by the PARCC Governing Board and (for the Algebra II, Integrated Mathematics III, and ELA/L grade 11 
assessments) the Advisory Committee on College Readiness as part of a post-policy reasonableness 
review. This group reviewed both the median threshold score recommendations from each committee 
and the variability in the threshold scores as represented by the standard error of judgment (SEJ) of the 
committee. Adjustments to the median threshold scores that were within 2 SEJ were considered to be 
consistent with the PLS panels’ recommendation. 


In addition to voting to adopt the performance standards based on the committees’ recommendations, 
this group also voted to conduct a shift in the PARCC performance levels to better meet the intended 
inferences about student performance. Holding the college- and career-ready (or on-track) expectations 
(i.e., the current level 4) constant, performance levels above this expectation were combined and 
performance levels below this expectation were expanded to create the final system of performance 
levels with three below and two above the college- and career-ready (or on-track) expectation. The shift 
in performance levels was accomplished using a scale anchoring process that involved two primary 
steps. In the first step, the top two performance levels, above college- and career-ready (or on-track), 
were combined into a single performance level and an additional performance level below college- and 
career-ready (or on-track) was created by empirically determining the midpoint between the existing 
two levels. In the second step, the performance level descriptors (PLDs) were updated using items that 
discriminated student performance well at this level to create a PLD aligned with the new empirically 
determined performance level. At this same time, PLDs for all performance levels were reviewed for 
consistency and continuity. Members of the original PLS committees were recruited to participate in this 
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process. Additional information about this process can be found in the Performance Level Setting 
Technical Report. 
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Section 12: Scale Scores 


PARCC assessments are designed to measure and report results in categories called master claims and 
subclaims. Master claims (or simply “claims”) are at a higher level than subclaims with content 
representing multiple subclaims contributing to each claim outcome. 


Four scale scores were reported for PARCC assessments.” A full summative scale score was reported for 
each mathematics assessment. A summative scale score and separate claim scores for Reading and 
Writing were reported for each English language arts/literacy (ELA/L) assessment. PARCC reports results 
according to five performance levels that delineate the knowledge, skills, and practices students are able 
to demonstrate: 


e Level 5: Exceeded expectations 

e Level 4: Met expectations 

e Level 3: Approached expectations 

e Level 2: Partially met expectations 

e Level 1: Did not yet meet expectations 


Subclaim outcomes describe student performance for content-specific subsets of the item scores 
contributing to a particular claim. For example, Written Expression and Knowledge of Conventions 
subclaim outcomes are reported along with Writing claim scores. Subclaim outcomes are reported as 
Below Expectations, Nearly Meets Expectations, or Meets or Exceeds Expectations. 


12.1 Operational Test Content (Claims and Subclaims) 


A claim is a statement about student performance based on how students respond to test questions. 
PARCC tests are designed to elicit evidence from students that supports valid and reliable claims about 
the extent to which they are college and career ready or on track toward that goal and are making 
expected academic gains based on the Common Core State Standards (CCSS). 


The number of items associated with each claim and subclaim outcome vary depending on test subject 
and grade. The item types vary in terms of the number of points associated with them, so that both the 
number of items and the number of points are important in evaluating the quality of a claim or subclaim 


score. 
12.1.1 English Language Arts/Literacy 


Table 12.1” includes the number of items and the number of points by subclaim and claim for ELA/L 
grade 3. Corresponding information is provided in Appendix 12.1 for all ELA/L grades. 


1 Addendum 12 presents a summary of results on scale scores for the fall 2016 administration. 
? Table A.12.1 in Appendix 12.1 is identical to Table 12.1. 
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Table 12.1 Form Composition for ELA/L Grade 3 


Claims Subclaims Number of Items Number of Points 
Reading 
Reading Literary Text 10-12 23-27 
Reading Informational Text 8-9 19-21 
Vocabulary 5-8 10-16 
Claim Total 26 58 
Writing 
Written Expression 2 27 
Knowledge of Conventions 1 9 
Claim Total 3 36 
SUMMATIVE TOTAL 29 94 


Note: Prose constructed responses (PCRs) consist of at least two writing traits (Written 
Expression and Writing Knowledge and Conventions) and, in some cases, a reading trait as well. 
An aggregated PCR item score is determined by summing the multiple scores the student 
received on two or three traits depending on the item. Therefore, each PCR trait is identified as 
a separate item in this table for the two writing subclaims and, in some cases, either the 
Reading Literary Text or the Reading Informational Text subclaim. 


Each ELA/L form contains items of varying types. The prose constructed-response (PCR) traits contribute 
to different claims and the aggregate of the traits contributes to the summative scale score. The 
following details the number of possible points and the associated subclaims for the three PCR tasks: 


e Literary Analysis Task 
e Research Simulation Task 
e = Narrative Writing Task 


The Literary Analysis Task and the Research Simulation Task are scored for two traits/subclaims: Reading 
Comprehension and Written Expression, and Knowledge of Conventions. The Narrative Writing Task is 
scored for two traits/subclaims: Written Expression and Knowledge of Conventions. All traits/subclaims 
are initially scored as either 0-3 or O—4; the Written Expression subclaims are multiplied by 3 (or 
weighted) to increase their contribution to the total score, making possible subclaim scores 0, 3, 6, and 
9, or 0, 3, 6, 9, and 12. The maximum possible points for ELA/L PCR items are provided in Table 12.2. 
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Table 12.2 Contribution of Prose Constructed-Response Items to ELA/L 
Possible Points 


Grade Score Literary Research Narrative Writing Task 
Analysis Simulation 
Task Task 

3 Reading 3 3 0 
Written Expression 9 9 9 
Knowledge of Conventions 3 3 3 
Total 15 15 12 
4-5 Reading 4 4 0 
Written Expression 12 12 9 
Knowledge of Conventions 3 3 3 
Total 19 19 12 
6-11 Reading 4 4 0 
Written Expression 12 12 12 
Knowledge of Conventions 3 3 3 
Total 19 19 15 


12.1.2 Mathematics 


Table 12.3 includes the numbers of items and points associated with subclaim scores for mathematics 
grade 3, as an example of the composition of the mathematics tests. 


Table 12.3 Mathematics Form Composition for Grade 3 


Subclaims Number of Items Number of Points 
Mathematics 
Major Content 26 30 
Additional & Supporting Content 10 10 
Expressing Mathematical Reasoning 4 14 
Modeling and Applications 3 12 
TOTAL 43 66 


Because there is substantial variation in the composition of the tests, corresponding information is 
provided in Appendix 12.1 tables for all mathematics grades/courses. 


13 Table A.12.10 in Appendix 12.1 is identical to Table 12.3. 
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12.2 Establishing the Reporting Scales 


PARCC reporting scales designate student performance into one of five performance levels“ with Level 
1 indicating the lowest level of performance and Level 5 indicating the highest level of performance. 
Threshold or cut scores associated with performance levels were initially expressed as raw scores on the 
performance level setting (PLS) forms approved by the PARCC Governing Board. 


A scale score task force was assembled by PARCC, which made recommendations about how threshold 
levels would be represented on the reporting scale. 


12.2.1 Full Summative Score Scale and Performance Levels 


There are 201 defined full summative scale score points for both ELA/L and mathematics, ranging from 
650 to 850. A scale score of 700 is associated with minimum Level 2 performance, and a scale score of 
750 is associated with minimum Level 4 performance. The threshold for summative performance levels 
on the scale score metric recommended by the scale score task force is described in Table 12.4. 


Table 12.4 Defined Summative Scale Scores 


Lowest Obtainable Highest Obtainable 
Level 2 Level 4 
Scale Score (LOSS) Scale Score (HOSS) 
Full Summative 650 700 750 850 


For spring 2015, scale scores were defined for each test as a linear transformation of the theta (82915) 
scale. The theta values associated with the Level 2 and Level 4 performance levels were identified using 
the test characteristic curve associated with the performance level setting form. With Levels 2 and 4 
scale scores fixed at 700 and 750, respectively, the relationship between theta (82915) and scale scores 
(Scale Scoreéz915) was established as 


Scale Scorezo15 = Azo15 * 92015 + B2015 


where Ajo15 is the slope and Bz915 is the intercept. The slope and intercept were established as 


750 — 700 


A2015 = a= ener 0 
2015_Level4 ~— Y%2015 Level 2 


and 


Boo15 = 750 — Azois * 92015 Level 4 


4 Section 11 provides an overview of the performance level setting process, and detailed information can be 
found in the Performance Level Setting Technical Report. 
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As indicated by these formulas, the slope and intercept for the summative scale scores were based on 
the theta scale, and by default the IRT parameter scale, established in 2015. Since the spring 2016 IRT 
parameter scale is the base scale for the IRT parameters, the scaling constants Az9;5 and Bj9;5 were 
updated in order to continue reporting performance levels, summative scale scores, claim scores, and 
subclaim performance levels on the same scale as 2015. Maintaining the 2015 scale allows for prior year 
scores to be compared to current and future scores, and it maintains the performance levels cut scores. 


New scaling constants for the summative scale score were needed for the linear transformation of the 
theta scale (82916) to the 2015 reporting scale (Scale Score 2915): 


Scale Scoreéz915 = SA2z016 * 92016 + SB2016 


The slope (slopeo15 to 2016) and intercept (interceptyo15 to 2016) generated during the year-to-year 
linking defined the linear relationship between the 2015 theta scale (69,5) and the 2016 theta scale 
(82916). These values were included in the scale score formula, and the formulas were used to solve for 
the slope (SA 91,6) and (SBz91,¢) intercept for 2016. 


The slope (Az91¢) was updated using the following formula: 


SA2016 = A2015/Slopezo15 to 2016) 


where A295 is the current scale score multiplicative constant, slopez915 to 2016 is the multiplicative 
coefficient from the year-to-year linking, and SA2916 is the scale score slope constant for 2016 and 
beyond. 


The intercept (B29;¢) was updated using the following formula: 


SB2016 = B2015 — A2o16 * interceptrzo15 to 2016) 


where Byo15 is the current scale score additive constant, Az91¢ is the updated scale score slope, and 
(SB2916) is the scale score intercept constant for 2016 and beyond. 


In addition, new scaling constants for the reading and writing claim scales were needed. The same 
formulas were applied by replacing the slope (Az9;5) and intercept (Bz 9,5) with the reading claim slope 
and intercept and the writing claim slope and intercept. 


A and B values resulting from these calculations as well as the theta values associated with the threshold 
performance levels are included in the 2015-2016 technical report. Also, the 2015-2016 technical 
report includes raw to scale score conversion tables for the performance level setting forms. 


12.2.2 ELA/L Reading and Writing Claim Scale 


There are 81 defined scale score points for Reading, ranging from 10 to 90. A scale score of 30 is 
associated with minimum Level 2 performance, and a scale score of 50 is associated with minimum Level 
4 performance. There are 51 defined scale score points for Writing, ranging from 10 to 60. A scale score 
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of 25 is associated with minimum Level 2 performance, and a scale score of 35 is associated with 
minimum Level 4 performance. The threshold Reading and Writing performance levels on the scale 
score metric recommended by the scale score task force are described in Table 12.5. 


Table 12.5 Defined Scaled Scores for Reading and Writing Claim Scores 


Lowest Obtainable Highest Obtainable 
Level 2 Level 4 
Scale Score Scale Score 
Reading 10 30 50 90 
Writing 10 25 35 60 


As with the full summative scores, scale scores for Reading and Writing were defined for each test as a 
linear transformation of the IRT theta (8) scale. The same IRT theta scale was used for Reading and 
Writing as was used for the ELA/L full summative scores. The theta values associated with the Level 2 
and Level 4 performance levels were identified using the test characteristic curve associated with the 
performance level setting form. As with the full summative scores, the relationship between theta and 
scale scores was established with Level 2 and Level 4 theta scores and the corresponding predefined 
scale scores. The formulas used for this are provided in Table 12.6. 


Table 12.6 Calculating Scaling Constants for Reading and Writing Claim Scores 


Reading Writing 
Scale = Ap*O+Bp Scale = Ay * 9+ By 
tes 50 — 30 po 35-25 
Srevet 4 — Prevel 2 Srevet 4 — Prevel 2 
Bp = 50 —A* Grevet 4 By = 35 — A * Breve 4 


A and B values resulting from these calculations are included in Appendix 12.2. 
12.2.3 Subclaims Scale 


The Level 4 cut is defined as Meets or Exceeds Expectations because high school students at Level 4 or 
above are likely to have the skills and knowledge to meet the definition of career and college readiness. 
The Level 3 cut is defined as Nearly Meets Expectations. Subclaim outcomes center on the Level 3 and 
Level 4 performance levels and are reported at three levels: 


e Below Expectations; 
e Nearly Meets Expectations; or 
e Meets or Exceeds Expectations. 


The subclaim performance levels are designated through the IRT theta (8) scale for the items associated 
with a particular subclaim. The theta values and corresponding raw scores associated with the Level 3 
and Level 4 performance levels were identified using the test characteristic curve. Students earning a 
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raw subclaim score equal to or greater than the Level 4 threshold were designated as Meets or Exceeds 
Expectations. Students not earning a raw subclaim score equal to or greater than the Level 3 threshold 
were designated as Below Expectations. Other students whose raw subclaim score fell between the 
Level 3 and 4 thresholds were designated as Nearly Meets Expectations. 


12.3 Creating Conversion Tables 


A PARCC conversion table relates the number of points earned by a student on the ELA/L full summative 
score, the mathematics full summative score, the Reading claim score, or the Writing claim score to the 
corresponding scale score for the test form administered to that student. An IRT inverse test 
characteristic curve (TCC) approach is used to develop the relationship between point scores and IRT 
ability estimates (Os). In carrying out the calculations, estimates of item parameters and thetas are 
substituted for parameters in the formulas in each step. 


Step 1: Calculate the expected item score (i.e., estimated item true score) for every scale score in the 
selected range (determined by LOSS, HOSS, and scale score increment) based on the generalized partial 
credit model for both dichotomous and polytomous items: 


M ,-1 


s,(0,)= > mp,,@,), (12-4) 


m=0 


m 


oxo) Da,(0,-5, + 4.) 


P im (8 ;) a -1 ¥ , (12-5) 


y =p Date = b+ 4) 


k=0 


where a,(0 ,—b, + d,,) = 0; s,(8 ,) is the expected item score for item ; ona scale score, !.; p,,,(9 ;) 


is the probability of a test taker with 0 getting score ,, on item ;; _ is the number of score categories of 
item , with possible item scores as consecutive integers fromOto | __, ; » isthe IRT scale constant 


(1.7); ajis a slope parameter; b; is a location parameter reflecting overall item difficulty; dj, is a location 
parameter incrementing the overall item difficulty to reflect the difficulty of earning score category k; v 
is the number of score categories. 


Step 2: Calculate the expected (weighted) test score for every scale score in the selected range: 
I 
YT, = = w,s,(8 ,) , (12-6) 
i=l 
where I, is the expected (weighted) test score ona scale score, ts _ isthe item weight for item ; (e.g., 


with w, = 2 ,a dichotomous item is scored as 0 or 2, and a three-category item is scored as 0, 2, or 4); 


, is the total number of items in a test form. 
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Step 3: Calculate the estimated conditional standard error of measurement (CSEM) for each scale score 
in the selected range: 


(12-7) 

L,(® ,) = (Da,)'[s,,(8 ,)- 5; (,)1, (12-8) 
5;,(8 ;) ~ > mp, (,), 

~t (12-9) 


where L,(8 ,) is the estimated item information function for item ; on scale score }.. 


Step 4: Match every raw score with a scale score. |; is the scale score for araw score if T, — 7, is 


minimum across all if S. 


Figure 12.1 contains TCCs, estimated CSEM curves, and estimated information (INF) curves for ELA/L 
grade 3. The curves in each figure are for the three core online forms and two core paper forms. The 
average difficulty of each form is reported and the curves are reported on the theta scale. Vertical 
dotted lines indicate the performance level cuts on the theta scale. For ELA/L grade 3, the three CBT and 
two PBT forms had very similar TCCs. CSEM and Information curves were similar for CBT forms and PBT 
forms. Appendix 12.3 contains TCC, CSEM, and INF curves for all ELA/L grades and all mathematics 
grades/courses. 


15 Grade 3 TCC, CSEM, and INF curves are also included in Appendix 12.4. 
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Figure 12.1 Test Characteristic Curves, Conditional Standard Error of Measurement Curves, and Information Curves for ELA/L Grade 3 
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12.4 Score Distributions 
12.4.1 Score Distributions for ELA/L 


All Students 

Figures 12.2 through 12.4 graphically represent the distributions of scale scores for grades 3 through 11 
ELA/L full summative, Reading, and Writing, respectively. The vertical axis of each graph, labeled 
“Density,” represents the proportion of students earning the scale score point indicated along the 
horizontal axis. 


The distributions of the ELA/L scale scores were fairly symmetrical and centered around the Level 4 cut 
score (i.e., 750), with the exception of grades 10 and 11, whose distributions were centered around the 
same location but whose shapes were slightly more irregular. 


March 27, 2018 Page 209 


PARCC 


2017 Technical Report 


March 27, 2018 


Grade 3 Grade 4 
0.010 
0.008 
0.008 
0.006 
Py 2 0.006 
@ a 
ral a 
0.004 
0.004 
0.002 
0.002 
0.000 — x 0.00 47 . ; : = 
650 700 750 800 850 650 700 750 800 850 
Surrrrative Scale Score Surrrrative Scale Score 
Grade 5 Grade 6 
0.0125 
o.oo 
0.0100 
0.008 
0.0075 
2 0.006 z 
a a 
0.0050 
0.004 
o.002 0.0025 
0.000 Lj 7 T T 71 0.0000 T 7 7 : 
650 700 750 800 850 650 700 750 800 850 
Surrrretive Scale Score Surrrrative Scale Score 
Figure 12.2 Distributions of ELA/L Scale Scores: Grades 3-11 
Page 210 


PARCC 


2017 Technical Report 


Grade 7 Grade 8 
ao10 0.010 
0.008 0.008 
0.006 0.006 
2 > 
z z 
g g 
0.004 0.004 
0.002 0.002 
0.000 T T T T T 0.000 T T T T r 
650 700 750 800 850 650 700 750 800 850 
Surrmative Scale Score Surrmative Scale Score 


Grade 9 Grade 10 
o.o10 0.008 
0.008 
0.006 
0.006 
2 2 
a a 
5 5 0004 
fal fal 
0.004 
0.002 
0.002 
0.000 T T T T T 0.000 T T T T T 
650 700 750 800 850 650 700 750 800 850 
Surrmative Scale Score Surrmative Scale Score 
Grade 11 
0.008 
0,006 
Py 
a 
s 
a 
0,004 
0,002 
0.000 
650 700 750 800 860 
Surrrrative Scale Score 


March 27, 2018 


Figure 12.2 (continued) Distributions of ELA/L Scale Scores: Grades 3-11 
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Reading scale scores tended to be centered between around or slightly below the Level 4 cut score of 50 
and were slightly more irregular than the summative scale scores. Distributions were fairly symmetric, 
with scores below the Level 4 cut score being a bit more common than higher scores. A portion of this is 
due to larger numbers of students earning near zero raw scores than near perfect raw scores. Raw score 
near zero can occur for a variety of reasons, such as student illness, and do not always indicate low skill 
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Figure 12.3 Distributions of Reading Scale Scores: Grades 3-11 
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Figure 12.3 (continued) Distributions of Reading Scale Scores: Grades 3-11 
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Writing scale score distributions were noticeably less smooth than Reading or ELA/L full summative 
distributions due to peaks related to the weighting of the Written Expression portion of the PCR tasks. 


The proportion of students earning the lowest obtainable scale score was fairly high for Writing. This 
occurred even though a score point of zero is typically necessary to obtain the minimum scale score. 
Writing items are embedded exclusively in PCR tasks, which tended to be difficult. The Written 
Expression trait/subclaim also tended to be the most difficult of the PCR traits. 
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Figure 12.4 Distributions of Writing Scale Scores: Grades 3-11 
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Figure 12.4 (continued) Distributions of Writing Scale Scores: Grades 3-11 


Groups 

Grade 3 group statistics for ELA/L full summative, Reading, and Writing scale scores are presented in 
Table 12.7.1 Mean scores were higher for female students relative to male students. Mean scores were 
highest for Asian students followed by white students and were lowest for American Indian/Alaska 
native students. Economically disadvantaged students performed less well than students who are not 
economically disadvantaged. English learners (EL) performed less well than non-EL students. Students 
with disabilities (SWD) performed less well than students without disabilities. 


Patterns of mean scale scores were quite similar in grades 4 through 8; corresponding tables for all 
grades are presented in Appendix 12.4. 


Table 12.7 Subgroup Performance for ELA/L: Grade 3 


Group Type Group N Mean SD Min Max 
Full Summative Score 420,496 738.23 41.04 650 850 
Gender Female 206,149 743.10 41.14 650 850 
Male 214,347 733.54 40.40 650 850 
Ethnicity American Indian/Alaska Native 5,920 716.34 34.04 650 850 
Asian 24,650 766.38 39.94 650 850 
Black or African American 73,056 721.93 38.49 650 850 
Hispanic/Latino 118,820 725.02 37.91 650 850 
Native Hawaiian or Pacific Islander 672 745.66 42.15 650 850 
Multiple Race Selected 15,036 743.81 40.77 650 850 
White 181,382 749.85 38.43 650 850 
Economic Status’ Economically Disadvantaged 209,280 722.50 37.11 650 850 


Not Economically Disadvantaged 210,597 753.81 38.74 650 850 
English Learner 


eetiie English Learner (EL) 61,177. 712.91 34.21 650 850 
Non-English Learner 349,974 742.44 40.55 650 850 
Disabilities Students with Disabilities (SWD) 61,999 709.57 38.69 650 850 
Students without Disabilities 357,072 743.21 39.37 650 850 
Reading Score 420,496 45.36 16.33 10 90 
Gender Female 206,149 46.67 16.26 10 90 
Male 214,347 44.10 16.31 10 90 
Ethnicity American Indian/Alaska Native 5,920 36.35 13.27 10 90 
Asian 24,650 55.60 15.92 10 90 
Black or African American 73,056 38.82 14.94 10 90 
Hispanic/Latino 118,820 39.74 14.69 10 90 
Native Hawaiian or Pacific Islander 672 47.29 16.35 10 90 


16 Table A.12.27 in Appendix 12.4 is identical to Table 12.7. 
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Group Type Group N Mean SD Min Max 
Multiple Race Selected 15,036 47.96 16.36 10 90 
White 181,382 50.37 15.55 10 90 
Economic Status’ — Economically Disadvantaged 209,280 38.98 14.45 10 90 
Not Economically Disadvantaged 210,597 51.69 15.62 10 90 
al Learner English Learner (EL) 61,177. 3491 1302 10 90 
Non-English Learner 349,974 47.09 16.17 10 90 
Disabilities Students with Disabilities (SWD) 61,999 3476 15.44 10 90 
Students without Disabilities 357.072 47.21 15.77 10 90 
Writing Score 420,496 30.41 12.53 10 60 
Gender Female 206,149 32.48 12.13 10 60 
Male 214,347 28.43 12.58 10 60 
Ethnicity American Indian/Alaska Native 5,920 25.71 11.67 10 60 
Asian 24,650 38.19 10.94 10 60 
Black or African American 73,056 26.50 12.48 10 60 
Hispanic/Latino 118,820 27.68 12.38 10 60 
Native Hawaiian or Pacific Islander 672 33.23 12.62 10 60 
Multiple Race Selected 15,036 31.22 12.46 10 60 
White 181,382 32.80 11.82 10 60 
Economic Status’ — Economically Disadvantaged 209,280 26.71 12.29 10 60 
shee Learner English Learner (EL) 61,177. 24.79 1201 10 60 
Non-English Learner 349,974 31.36 12.36 10 60 
Disabilities Students with Disabilities (SWD) 61,999 21.99 12.27 10 60 
Students without Disabilities 357,072 31.88 11.98 10 60 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP)—receipt of 
free or reduced-price lunch (FRL). 


Grade 9 group statistics for ELA/L, Reading, and Writing scale scores are presented in Table 12.8. Mean 
scores were very similar to what was observed for grades 3 through 8. Mean scores were higher for 
female students than for male students. Mean scores were highest for Asian students followed by white 
students; scores were lowest for American Indian/Alaska native students. Economically disadvantaged 
students performed less well than students who are not economically disadvantaged. English learners 
(EL) performed less well than non-EL students. Students with disabilities (SWD) performed less well than 
students without disabilities. 


Similar patterns are observed in other grades, with some small variations in the ordering of the ethnic 
groups. For example, in ELA/L grade 11, American Indian/Alaska native students had higher mean scores 
than black or African American students, and native Hawaiian or Pacific Islander students and students 
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who selected multiple races scored higher than white students. Corresponding tables for grades 10 and 


11 are presented in Appendix 12.5. 


Table 12.8 Subgroup Performance for ELA/L: Grade 9 


Group Type Group 
Full Summative Score 
Gender Female 
Male 
Ethnicity American Indian/Alaska Native 


Economic Status" 


English Learner 
Status 


Disabilities 


Reading Score 
Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 
Gender 
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Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non-English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non-English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 


N 
188,201 
90,951 
97,250 
3,407 
12,192 
21,038 
62,335 
373 
3,839 
84,432 
78,341 


109,453 
15,584 


166,433 
30,113 
157,483 
188,201 
90,951 
97,250 
3,407 
12,192 
21,038 
62,335 
373 
3,839 
84,482 
78,341 


109,453 


15,584 


166,433 
30,113 
157,483 
188,201 
90,951 


Mean 
740.92 
748.89 
733.47 
719.95 
771.45 
727.01 
726.81 
749.01 
744.07 
751.22 
725.35 


752.17 


698.36 


745.00 

713.88 

745.20 
46.54 
48.72 
44.50 
37.37 
57.70 
40.82 
40.66 
49.35 
48.46 
51.02 
40.14 


51.15 


29.33 


48.14 
36.22 
48.55 
30.89 
34.13 


SD 
37.97 
36.81 
37.53 
30.27 
37.59 
34.98 
35.14 
37.05 
37.68 
34.90 
34.39 


36.42 


26.51 


36.44 
34.03 
36.43 
15.34 
14.97 
15.40 
12.13 
15.49 
13.91 
13.92 
14.90 
15.45 
14.25 
13.66 


14.81 


10.03 


14.75 
13.79 
14.81 
11.86 
10.64 


Max 
850 
850 
850 
828 
850 
850 
850 
850 
850 
850 
850 


850 
846 


850 

850 

850 
90 
90 
90 
79 
90 
90 
90 
90 
90 
90 
90 


90 


90 


90 
90 
90 
60 
60 
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Group Type Group N Mean SD Min Max 
Male 97,250 27.86 12.14 10 60 
Ethnicity American Indian/Alaska Native 3,407. 26.58 10.85 10 56 
Asian 12,192 39.46 9.95 10 60 
Black or African American 21,038 27.41 11.76 10 60 
Hispanic/Latino 62,335 27.42 11.84 10 60 
Native Hawaiian or Pacific Islander 373 33.33 11.47 10 60 
Multiple Race Selected 3,839 31.02 11.84 10 60 
White 84,482 33.28 10.92 10 60 
Economic Status’ — Economically Disadvantaged 78,341 26.95 11.76 10 60 
Not Economically Disadvantaged 109,453 33.76 11.07 10 60 
— Learner English Learner (EL) 15,584 19.49 10.69 10 60 
Non-English Learner 166,483 32.04 11.37 10 60 
Disabilities Students with Disabilities (SWD) 30,113 22.92 11.79 10 60 
Students without Disabilities 157,483 32.45 11.23 10 60 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP)—receipt of 
free or reduced-price lunch (FRL). 


12.4.2 Score Distributions for Mathematics 


All Students 
Figure 12.5 graphically represents the distributions of scale scores for grades 3 through 8 mathematics. 
Scale score distributions peaked between 700 and the Level 4 performance level cut of 750. 
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Figure 12.5 Distributions of Mathematics Scale Scores: Grades 3-8 
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Figure 12.6 graphically represents the distributions of scale scores for Algebra |, Geometry, Algebra Il, 
and Integrated Mathematics |, Il, and III. Scale score distributions peaked between 700 and the 750 
Level 4 performance level cut score, with the exception of Integrated Mathematics II, which peaked 
between 750 and 775. Distributional shapes were somewhat similar for Algebra | and Integrated 
Mathematics |, for Geometry and Integrated Mathematics II, and for Algebra II and Integrated 
Mathematics Ill. 
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Grade 3 group statistics for mathematics scale scores are presented in Table 12.9.7” Mean scores were 
slightly higher for female students relative to male students. Mean scores were highest for Asian 
students followed by white students and were lowest for American Indian/Alaska native students. 
Economically disadvantaged students performed less well than students who are not economically 
disadvantaged. English learners (EL) performed less well than non-EL students. Students with disabilities 
(SWD) performed less well than students without disabilities. 


Table 12.9 Subgroup Performance for Mathematics Scale Scores: Grade 3 


Group Type Group N Mean SD Min Max 
Full Summative Score 425,744 742.22 37.08 650 850 
Gender Female 208,739 742.47 35.83 650 850 
Male 217,005 741.98 38.25 650 850 
Ethnicity American Indian/Alaska Native 5,920 724.13 31.47 650 845 
Asian 25,182 772.99 36.36 650 850 
Black or African American 73,166 726.20 34.03 650 850 
Hispanic/Latino 122,975 730.60 33.67 650 850 
Native Hawaiian or Pacific Islander 684 746.93 38.61 650 850 
Multiple Race Selected 15,061 745.90 37.32 650 850 
White 181,746 752.53 34.61 650 850 
Economic Status’ — Economically Disadvantaged 213,460 728.31 33.58 650 850 


Not Economically Disadvantaged 211,662 756.19 35.11 650 850 
English Learner 


eerie English Learner (EL) 66,662 723.87 33.05 650 850 
Non-English Learner 349,729 745.55 36.74 650 850 
Disabilities Students with Disabilities (SWD) 62,252 718.87 37.60 650 850 
Students without Disabilities 362,041 746.23 35.47 650 850 


Note: Economic status was based on participation in National School Lunch Program (NSLP)—receipt of 
free or reduced-price lunch (FRL). 


Generally similar patterns were observed in other grades, with some slight variations in the orderings of 
the ethnic groups. For example, American Indian/Alaska native students had slightly higher mean scores 
than black or African American students for some assessments. Corresponding tables for all grades are 
presented in Appendix 12.5. 


Algebra | scale score statistics are presented in Table 12.10. Mean scores were higher for female 
students relative to male students. Mean scores were highest for Asian students followed by white 
students and were lowest for American Indian/Alaska native students. Economically disadvantaged 
students performed less well than students who are not economically disadvantaged. English learners 


17 Table A.12.36 in Appendix 12.5 is identical to Table 12.9. 
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(EL) performed less well than non-EL students. Students with disabilities (SWD) performed less well than 
students without disabilities. 


Table 12.10 Subgroup Performance for Mathematics Scale Scores: Algebra | 


Group Type Group N Mean SD Min Max 
Full Summative Score 260,806 738.16 34.95 650 850 
Gender Female 126,562 739.22 33.69 650 850 
Male 134,244 737.17. 36.07. 650 850 
Ethnicity American Indian/Alaska Native 3,641 716.02 25.07 650 821 
Asian 18,079 770.12 36.40 650 850 
Black or African American 45,085 722.83 28.93 650 850 
Hispanic/Latino 74,965 72445 29.81 650 850 
Native Hawaiian or Pacific Islander 452 746.57 35.29 650 850 
Multiple Race Selected 6,580 743.53 35.41 650 850 
White 111,446 748.89 32.87 650 850 
Economic Status’ — Economically Disadvantaged 106,203 723.39 29.35 650 850 


Not Economically Disadvantaged 153,948 748.42 34.85 650 850 
English Learner 


Status English Learner (EL) 20,325 707.78 24.46 650 850 
Non-English Learner 235,967 740.64 34.45 650 850 

“Disabilities «Students with Disabilities (SWD) 40,764 717.33 30.34 650 850 — 
Students without Disabilities 219,187 742.10 34.37 650 850 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP)—receipt of 
free or reduced-price lunch (FRL). 


Similar patterns were observed in the other high school tests with some of the previously mentioned 
exceptions in the ordering of the ethnicities applying to these tests as well. In Algebra II, males 
outperformed females by a small margin. Corresponding tables are presented in Appendix 12.5. 


Integrated Mathematics | scale score statistics are presented in Table 12.11. Mean scores were higher 
for female students relative to male students. Mean scores were highest for native Hawaiian or Pacific 
Islander students followed by Asian students, and were lowest for American Indian/Alaska native 
students. Economically disadvantaged students performed less well than students who are not 
economically disadvantaged. English learners (EL) performed less well than non-EL students. Students 
with disabilities (SWD) performed less well than students without disabilities. 
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Table 12.11 Subgroup Performance for Mathematics Scale Scores: Integrated Mathematics | 


Group Type Group N Mean SD Min Max 
Full Summative Score 10,964 730.40 33.72 650 850 
Gender Female 5,095 730.86 32.94 650 841 
Male 5,869 730.01 34.38 650 850 
Ethnicity American Indian/Alaska Native 149 716.07. 26.71 650 790 
Asian 431 745.92 39.15 650 850 
Black or African American 809 721.63 30.23 650 821 
Hispanic/Latino 5,272 720.80 29.02 650 835 
Native Hawaiian or Pacific Islander 28 748.39 42.15 673 810 
Multiple Race Selected 272 734.92 33.37 650 850 
White 3,997 743.29 34.67 650 850 
Economic Status’ — Economically Disadvantaged 5,793 721.21 29.68 650 838 


Not Economically Disadvantaged 5,164 740.77 34.94 650 850 
English Learner 


Sets English Learner (EL) 2,191 707.58 21.98 650 815 
Non-English Learner 6,506 734.82 32.28 650 850 
Disabilities Students with Disabilities (SWD) 1,356 709.29 28.29 650 830 
Students without Disabilities 9,601 733.42 33.34 650 848 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP)—receipt of 
free or reduced-price lunch (FRL). 


Somewhat similar patterns were observed in Integrated Mathematics Il and Integrated Mathematics III, 
but sample sizes for some subgroups are very small, and caution should be used in interpretations. 
Tables for these tests can be found in Appendix 12.5. 


12.5 Interpreting Claim Scores and Subclaim Scores 


12.5.1 Interpreting Claim Scores 


PARCC ELA/L assessments provide separate claim scale scores for both Reading and Writing. The claim 
scale scores and the summative scale score are on different scales; therefore, the sum of the scale 
scores for each claim will not equal the summative scale score. PARCC Reading scale scores range from 
10 to 90 and PARCC Writing scale scores range from 10 to 60. 


The claim scores can be interpreted by comparing a student’s claim scale score to the average 
performance for the school, district, and state. The PARCC Individual Student Report (ISR) provides the 
student scale score results and the average scale score results for the school, district, and state. 


12.5.2 Interpreting Subclaim Scores 


Within each reporting category are specific skill sets (subclaims) students demonstrate on the PARCC 
assessments. Subclaim categories are not reported using scale scores or performance levels. Subclaim 
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performance for PARCC assessments is reported using graphical representations that indicate how the 
student performed relative to the Level 3 and Level 4 performance levels for the content area. 


Subclaim indicators represent how well students performed in a subclaim category relative to Level 3 
and Level 4 thresholds for the items associated with the subclaim category. To determine a student’s 
subclaim performance, the Level 3 and Level 4 thresholds corresponding to the IRT based performance 
for the items for a given subclaim determined the reference points for Approached Expectations and Did 
Not Yet Meet Expectations or Partially Met Expectations, respectively. 


Student performance for each subclaim is marked with a subclaim performance indicator. 


e = An‘up’ arrow for the specified subclaim for Meets or Exceeds Expectations indicates that the 
student’s performance for the subclaim was equal to or better than the threshold for Level 4 
(i.e., students whose summative scale score was 750). 

e =A ‘bidirectional’ arrow for the specified subclaim for Nearly Meets Expectations indicates that 
the student’s performance was below the Level 4 threshold (i.e., summative scale score was 
750) but better than or equal to the Level 3 threshold (i.e., summative scale score was 725). 

e A ‘down’ arrow for the specified subclaim for Below Expectations indicates that the student’s 
performance for the subclaim was below the Level 3 threshold (i.e., summative scale score was 
725). 
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Section 13: Student Growth Measures 


Student growth percentiles (SGPs) are normative measures of annual progress. Normative measures are 
useful in answering questions like “How does my academic progress compare with the academic 
progress of my peers?” In contrast to criterion-referenced measures of growth, which describe 
academic growth toward a particular goal, norm-referenced measures of growth describe students’ 
growth relative to that of students who performed similarly in the past (Betebenner, 2009). 


SGPs measure individual student progress by tracking student scores from one year to the next. SGPs 
compare a student’s performance to that of his or her academic peers both within the state and across 
the consortium. Academic peers are defined as students in the norm group who took the same PARCC 
assessment as the student in prior years and achieved a similar score. 


The SGP describes a student’s location in the distribution of current test scores for all students who 
performed similarly in the past. SGPs indicate the percentage of academic peers above whom the 
student scored. With a range of 1 to 99, higher numbers represent higher growth and lower numbers 
represent lower growth. For example, a SGP of 60 on grade 7 ELA/L means that the student scored 
better than 60 percent of the students in the state or consortium who took grade 7 ELA/L in spring 2017 
and who had achieved a similar score as this student on the grade 6 ELA/L assessment in spring 2016 
and the grade 5 ELA/L assessment in spring 2015.18 A SGP of 50 represents typical (median) student 
growth for the state or consortium. Because students are only compared with other students who 
performed similarly in the past, all students, regardless of starting point, can demonstrate high or low 
growth. 


The 2016-2017 academic year is the third year of PARCC test administration, allowing for the calculation 
of SGPs using two prior year scores, if available. Students in states that participated in the consortium in 
spring 2015 and spring 2016 generally received SGPs based on two prior scores. Students in states that 
participated in the consortium in spring 2016 received SGPs based on one prior score. Students who do 
not have a previous PARCC test score, which include any new testers and all grade 3 students, do not 
receive an SGP. 


13.1 Norm groups 


The norm groups consisted of students with the same prior scores based on grade or content area 
progressions (academic peers). SGPs were based on up to two years of prior test scores from spring 
2015 and spring 2016 PARCC administrations. States administering traditional mathematics PARCC 
assessments in fall 2015 or fall 2016 may also have SGPs based on these prior scores. Tables 13.1—-13.8 


18 Note: Because regression modeling is used to establish the relationship between prior and current scores, the 
SGP is for students with the exact same prior scores. This often leads to confusion among non-technical 
stakeholders who often ask, “How many students are there with exactly the same prior scores?” To avoid 
explaining regression to non-technical stakeholders, the “similar scores” is often used to finesse the idea of 
regression without mentioning it. 
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list the grade or content area progressions required for SGPs based on one prior or two prior test scores 
for ELA/L grades 3 through 11, mathematics grades 3 through 8, Algebra |, Geometry, Algebra II, 
Integrated Mathematics |, Il, and Ill, respectively. In general, the progressions of grade levels and 
content areas are consecutive. The traditional and integrated mathematics courses have progressions 
that are not consecutive but reflect student progression for high school mathematics courses. SGPs 
were calculated for all norm groups with at least 1,000 students. Some progressions did not meet the 
minimum sample size for SGP calculations. 


Table 13.1 ELA/L Grade-Level Progressions for One-year and Two-year Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 
N/A N/A Grade 3* 

N/A Grade 3 Grade 4 

Grades 3 and 4 Grade 4 Grade 5 

Grades 4and5 Grade 5 Grade 6 

Grades 5 and 6 Grade 6 Grade 7 

Grades 6 and 7 Grade 7 Grade 8 

Grades 7 and 8 Grade 8 Grade 9 

Grades 8 and 9 Grade 9 Grade 10 

Grades 9 and 10 Grade 10 Grade 11 


*SGP not calculated for grade 3 since there are no prior scores. 


Table 13.2 Mathematics Grade-Level Progressions for One-year and Two-year Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 
N/A N/A Grade 3* 

N/A Grade 3 Grade 4 

Grades 3 and 4 Grade 4 Grade 5 

Grades 4 and 5 Grade 5 Grade 6 

Grades 5 and 6 Grade 6 Grade 7 

Grades 6 and 7 Grade 7 Grade 8 


*SGP not calculated for grade 3 since there are no prior scores. 


Table 13.3 Algebra | Grade/Content Area Progressions for One-year and Two-year Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 
Grades 5 and 6 Grade 6 Algebra | 

Grades 6 and 7 Grade 7 Algebra | 

Grades 6 or 7 and8 Grade 8 Algebra | 

Grades 6, 7, or 8 and Geometry Algebra | 

Geometry 

Grade 8 and Integrated Math! Integrated Mathematics | Algebra | 

Integrated Mathematics | and Integrated Mathematics II Algebra | 


Integrated Mathematics II 
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Table 13.4 Geometry Grade/Content Area Progressions for One-year and Two-year Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 
Grades 5 and 6 Grade 6 Geometry 

Grades 6 and 7 Grade 7 Geometry 

Grades 6 or 7 and8 Grade 8 Geometry 

Grades 6,7, or8andAlgebral Algebra | Geometry 

Grade 8 and Integrated Integrated Mathematics | Geometry 

Mathematics | 

Integrated Mathematics | and Integrated Mathematics II Geometry 


Integrated Mathematics II 


Table 13.5 Algebra II Grade/Content Area Progressions for One-year and Two-year Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 
Grades 6 and 7 Grade 7 Algebra II 

Grades 7 and 8 Grade 8 Algebra Il 

Grades 7 or 8 and Algebra | Algebra | Algebra Il 

Grade 8 or Algebra | and Geometry Algebra II 

Geometry 

Grade 8 and Integrated Integrated Mathematics | Algebra Il 

Mathematics | 

Integrated Mathematics | and Integrated Mathematics II Algebra II 


Integrated Mathematics II 


Table 13.6 Integrated Mathematics | Grade/Content Area Progressions for One-year and Two-year 
Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 
Grades 5 and 6 Grade 6 Integrated Mathematics | 
Grades 6 and 7 Grade 7 Integrated Mathematics | 
Grades 6 or 7 and 8 Grade 8 Integrated Mathematics | 
Grades 7 or 8 and Algebra | Algebra | Integrated Mathematics | 
Grade 8 or Algebra | and Geometry Integrated Mathematics | 
Geometry 


Table 13.7 Integrated Mathematics I] Grade/Content Area Progressions for One-year and Two-year 
Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 

Grades 6 and 7 Grade 7 Integrated Mathematics II 
Grades 7 and 8 Grade 8 Integrated Mathematics II 
Grades 7 or 8and Integrated Algebral Integrated Mathematics II 


Mathematics | 
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Table 13.8 Integrated Mathematics Il Grade/Content Area Progressions for One-year and Two-year 
Prior Test Scores 


Two Prior Year Test Scores One Prior Year Test Score Current Year Test Score 
Grades 6 and 7 Grade 7 Integrated Mathematics III 
Grades 7 and 8 Grade 8 Integrated Mathematics III 
Grades 7 or 8and Integrated Algebral Integrated Mathematics III 
Mathematics | 

Integrated Mathematics! and Integrated Mathematics II Integrated Mathematics III 


Integrated Mathematics II 


13.2 Student Growth Percentile Estimation 


SGPs are calculated using quantile regression, which describes the conditional distribution of the 
response variable with greater precision than traditional linear regression, which describes only the 
conditional mean (Betebenner, 2009). This application of quantile regression uses B-spline smoothing to 
fit a curvilinear relationship between a norm group’s prior and current scores. Cubic B-spline basis 
functions are used when calculating SGPs to better model the heteroscedasticity, nonlinearity, and 
skewness in assessment data. 


For each group, the quantile regression fits 100 relationships (one for each percentile) between 
students’ prior and current scores. The result is a single coefficient matrix that relates students’ prior 
achievement to their current achievement at each percentile. The National Center for the Improvement 
of Educational Assessment (NCIEA) performed the analyses using Betebenner’s (2009) non-linear 
quantile-regression based SGP. The analysis was done in the SGP package in R (Betebenner, Van 
Iwaarden, Domingue, & Shang, 2017). For details on student growth percentiles, see Betebenner’s A 
Technical Overview of the Student Growth Percentile Methodology: Student Growth Percentiles and 
Percentile Growth Projections/Trajectories (2011). 


Betebenner’s (2009) SGP model uses Koenker’s (2005) quantile regression approach to estimate the 
conditional density associated with a student’s score at administration t conditioned on the student’s 
prior score(s). Quantile regression functions represent the solution to a loss function much like least 
squares regression represents the solution to a minimization of squared deviations. The conditional 
quantile functions are parametrized as a linear combination of B-spline basis functions (Wei and He, 


2006) to smooth irregularities found in the data. For PARCC scores from administration t (where t 22), 


the tth quantile function for Y; conditional on prior scores (Yt-1,...,Y1) is 


Qye (T| Yeu ---.¥1) = D525 Dy Diy (%) 8), 


where Qj, (i=1,2,..., n students; j=1, ..., t-1 administrations) represent the B-spline basis functions. The 
SGP of each student / is the midpoint between the two consecutive Ts whose quantile scores capture the 
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student’s current score, multiplied by 100. For example, a student with a current score that lies between 
the fitted value for tT = .595 and t = .605 would receive a SGP of 60. 


SGPs are assumed to be uniformly distributed and uncorrelated with prior achievement. Scale score 
conditional standard errors of measurement (CSEMs) were incorporated for calculation of SGP standard 
errors of measurement (SEMs). Goodness of fit results were checked (i.e., uniform distribution of SGPs 
by prior achievement) for indications of ceiling/floor effects for each SGP norm-group analysis. 


13.3 Student Growth Percentile Results/Model Fit for Total Group 


The estimation of SGPs was conducted for each student who had at least one prior score. Each analysis 
is defined by the norm cohort group (grade/sequence). A goodness of fit plot is produced for each 
analysis run in 2017. A ceiling/floor effects test identifies potential problems at the highest obtainable 
scale scores (HOSS) and lowest obtainable scale scores (LOSS). Other fit plots compare the observed 
conditional density of SGP estimates with the theoretical uniform density. If there is perfect model fit, 
10 percent of the estimated growth percentiles are expected within each decile band. A Q-Q plot 
compares the observed distribution with the theoretical distribution; ideally the step function lines do 
not deviate much from the ideal line of perfect fit. 


Tables 13.9 and 13.10 summarize SGP estimates for the total testing group for ELA/L and mathematics, 
respectively. SGPs were calculated at the consortium level and, if sample size was sufficient, the state 
level. Consortium-based median SGPs ranged from 49 to 51, with most having a median of 50. If the 
model is a perfect fit, the median is expected to be 50 with norm-referenced data. The minimum SGP is 
1 and the maximum SGP is 99. The average standard error for the consortium-based SGPs is within 
expectations for these models. 


In general, SGPs can be divided into three categories: below 30 indicating that a student is not meeting a 
year’s worth of growth, a SGP of 30-70 indicating that a student did achieve a year’s worth of growth, 
and a SGP over 70 indicating that the student surpassed a year’s worth of growth. It is important to note 
that definitions such as these are not inherent to the SGP method, but rather require expert judgment 
(Betenbenner, 2009). The observed standard errors, ranging from 12—16, support these interpretations 
(Betenbenner et al., 2016). 


March 27, 2018 Page 231 


PARCC 2017 Technical Report 


Table 13.9 Summary of ELA/L SGP Estimates for Total Group 


Grade Sample Average Average Median 
Level Size SGP Standard Error SGP 
4 398,880 49.99 13.05 50 
5 391,394 50.05 14.26 50 
6 382,695 50.01 13.12 50 
7 381,508 49.95 13.44 50 
8 374,408 49.95 13.64 50 
9 160,493 49.95 13.37 50 
10 102,861 49.55 13.10 49 
11 82,079 49.58 11.77 49 


Table 13.10 Summary of Mathematics SGP Estimates for Total Group 


Grade Sample Average Average Median 
Level Size SGP Standard Error SGP 
4 400,115 50.03 12.87 50 
5 392,070 50.05 14.47 50 
6 382,465 50.02 14.39 50 
7 367,870 50.02 15.93 50 
8 288,571 49.94 16.27 50 
Al 205,209 50.33 15.67 50 
GO 117,261 51.01 14.17 51 
A2 91,238 49.56 16.00 49 
M1 8,549 49.75 16.45 50 
M2 -- -- -- -- 
M3 os = a = 
Note: “--” indicates insufficient sample for SGP calculation for these tests. 


13.4 Student Growth Percentile Results for Subgroups of Interest 


Median SGPs are provided for subgroups of interest. With norm-referenced data, the median of all SGPs 
is expected to be close to 50. Median subgroup growth percentiles below 50 represent growth lower 
than the consortium median, and median growth percentiles above 50 represent growth higher than the 
consortium median. Table 13.11 summarizes SGPs for groups of interest for ELA/L grade 4. The ELA/L 
tables for grades 5-11 are provided in the Appendix (Tables A.13.1 — A.13.8). Table 13.12 summarizes 
SGPs for groups of interest for mathematics grade 4; the other mathematics subgroup results are 
provided in the Appendix (Tables A.13.9 — A.13.17). All median SGPs for subgroups of interest fell within 
the band of 30-70, which is considered to be adequate growth. 
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Gender 


English Language Arts/Literacy 


The median SGP for females tended to be 4 to 7 points higher than the median SGPs for males, on 
average. The standard error for males and females were comparable to the total group. 


Mathematics 

There was no consistent pattern between median SGPs for females and males, with the median SGPs 
within two points, on average. The standard errors were similar to the total group. Integrated 
Mathematics II and Ill had insufficient sample size for SGP subgroup results to be reported. 


Ethnicity 

English Language Arts/Literacy 

The African American group median SGP ranged between 45 and 50, with students in higher grades at 
the higher range. Asian/Pacific Islanders tended to have the highest median SGPs, over 60 for all tests 
but grade 11. American Indian/Alaska Native had a wide range in median SGPs across grades, with the 
lowest in grade 4 of 38 and the highest in grade 11 of 60. The median SGP for Hispanics and multiple 
ethnicities was around 50, plus or minus 3 points. The standard errors were similar to that of the total 


group. 


Mathematics 


The median SGP for African Americans ranged from 41 to 51, with the highest growth on the Integrated 
Mathematics | test. Asian/Pacific Islanders had the highest SGPs across all tests, with a minimum of 55 
and a maximum of 62. American Indian/Alaska Native had higher median SGPs on the traditional 
mathematics tests, on average, than on grades 3 through 8; the minimum SGP was 39 and the maximum 
SGP was 58. The median SGP for Hispanics and multiple ethnicities varied slightly from 50. For all 
ethnicities, the standard errors for all groups were under 19 points. 


Special Instructional Needs 

English Arts/Literacy 

Economically disadvantaged students showed similar or slightly lower median growth than students 
who were not economically disadvantaged. Except for grade 11, English learners had median SGPs up to 
five points lower than the total population. In grade 11, the median SGP was 52 for English learners. 
Students with disabilities tended to show lower growth compared to students without disabilities, with 
the minimum observed median SGP of 39 in grade 3 and the maximum observed median SGP of 45 in 
grades 9 and 10. The standard errors for these groups were similar to those observed for the total 


group. 


Mathematics 

Economically disadvantaged students tended to have lower median SGPs than those who were not 
economically disadvantaged, with medians up to five points below 50. Similarly, English learners tended 
to have median SGPs below the consortium average, with a range of 41-48. The range for students with 
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disabilities is 43-47. The standard errors for special education students tended to be slightly higher than 
the total group, but below 20 points. 


Students Taking Spanish Forms 

Mathematics 

There was a wide range of median growth percentiles for students taking Spanish forms. The number of 
students in this group tended to be smaller, on average. Twenty-seven students taking Integrated 
Mathematics | on a Spanish form received an SGP. This test had the highest median SGP for Spanish 
forms of 58. In Algebra II, 775 students received a SGP. Algebra II had the lowest SGP median of 34. 
Typically, the Spanish form median SGP was within 5 points of 50. These forms had a higher standard 
error on average, likely due to lower sample sizes in the norm groups relative to the total population. 
Nonetheless, the highest average standard error for this group was just over 21. 


Table 13.11 Summary of SGP Estimates for Subgroups: Grade 4 ELA/L 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 203,109 48.22 13.13 47 

Female 195,771 51.83 12.97 53 
Ethnicity 

White 174,974 5227 12.88 53 

African American 67,347 44.70 13.24 43 

Asian/Pacific Islander 23,183 59.11 12.56 63 

American Indian/Alaska Native 5,114 41.26 13.36 38 

Hispanic 113,794 48.13 13.29 47 

Multiple 13,626 49.88 13.00 50 
Special Instructional Needs 

Economically Disadvantaged 195,933 46.12 13.27 45 

Not-economically Disadvantaged 202,947 BS,/2 12.84 55 

English Learner (EL) 34,649 45.78 13.91 44 

Non English Learner 364,231 50.39 12.97 51 

Students with Disabilities (SWD) 62,933 42.50 13.56 39 

Students without Disabilities 335,947 51.39 12.96 52 
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Table 13.12 Summary of SGP Estimates for Subgroups: Grade 4 Mathematics 


2017 Technical Report 


Total Average 
Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 203,729 50.77 12:87 51 

Female 196,386 49.26 12.88 49 
Ethnicity 

White 175,093 52.78 1261 54 

African American 67,242 44.15 13.24 42 

Asian/Pacific Islander 23,477 58.94 12.86 62 

American Indian/Alaska Native 5,093 43.69 13.24 41 

Hispanic 114,750 47.67 13.06 47 

Multiple 13,613 50.2 12.90 50 
Special Instructional Needs 

Economically Disadvantaged 196,903 46.29 13.10 45 

Not-economically Disadvantaged 203,212 53.65 12.66 55 

English Learner (EL) 35,923 45.63 13.88 44 

Non English Learner 364,192 50.46 12.78 51 

Students with Disabilities (SWD) 62,799 45.94 13.77 44 

Students without Disabilities 337,316 50.79 12.71 51 
Spanish Language Form 2,628 44.32 13.70 42 
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Section 14: Quality Control Procedures 


Quality control in a testing program is a comprehensive and ongoing process. This section describes 
procedures put into place to monitor the quality of the item bank, test form, and ancillary material 
development. The quality checks for scanning, image editing, scoring, and data screening during 
psychometric analyses are also outlined. Additional quality information can be found in the PARCC 
Program Quality Plan document. 


14.1 Quality Control of the Item Bank 


The PARCC summative item bank consists of test passages and items, their associated metadata, and 
status (e.g., operational-ready, field-test ready, released, etc.). The items on the 2016-2017 
assessments were developed by Pearson and West Ed and put in the item bank once created. 


The ABBI bank houses the passages and items, art, associated metadata, rubrics, alternate text for use 
on accommodated forms, and text complexity documentation. It provides an item previewer that allows 
items to be viewed and interacted with in the same way students see and interact with items and tools, 
and manages versioning of items with a date/time stamp. It allows PARCC reviewers to vote on item 
acceptance, and to record and retain their review notes for later reconciliation and reference. PARCC 
item and passage review committee participants conducted their review in the item banking system. 
The committee members viewed the items as the student would, and could vote to alter the item, 
accept or reject the item, and record their comments in the system. After each meeting, reports were 
forwarded to Parcc Inc. The reports were generated by the item banking system and summarized 
feedback from the committee reviewers. 


All new development for the PARCC assessments is being created within the ABBI system, which 
employs templates to control the consistency of the underlying scoring logic and QTI creation for each 
item type. The ABBI system incorporates a previewer that allows the PARCC reviewers to validate the 
content of the item and validate the expected scoring of tasks. It supports the full range of PARCC 
review activities, including content review, bias and sensitivity review, expert editorial review, data 
review, and test construction review. It provides insight into the item edit process through versioning. A 
series of metadata validations at key points in the development cycle provide support for metadata 
consistency. The bank can be queried on the full range of PARCC metadata values to support bank 
analysis. 


14.2 Quality Control of Test Form Development 


Test forms were built based upon targets and the established blueprints set by PARCC. The construction 
process started with specification and requirement capture to create the test specification document. 
From there items were pulled into forms based on the criteria approved in the test specifications 
document. After forms composition, the forms went through a review process that involved groups 
from Pearson and the PARCC states. Quality control steps were conducted on the items and forms 
evaluating several item characteristics (e.g., content accuracy, completeness, style guide conformity, 
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tools function). Revisions were incorporated into the forms before final review and approval. Section 2.2 
provides more details on the form development process. 


The forms quality assurance was performed by Pearson’s Assessment and Information Quality (AlQ) 
organization. AlQ completed a comprehensive review of all online forms for the PARCC administration 
cycle. This group is part of Pearson’s larger Organizational Quality group and operates exclusively to 
validate form operability. The group validates that the functionality of every online form is working to 
specifications. The overall functionality and maneuverability of each form is checked, and the behavior 
of each item within the form is verified. (Quality processes for paper forms are described in section 
14.3.) 


The items within each form were tested to verify that they operated as expected for test takers. As a 
further aspect of the testing process, AlQ confirmed that forms were loaded correctly and that the audio 
was correct when compared to text. Sections and overviews were reviewed. Technology-enhanced 
items also were tested as an additional measure. As enumerated in the Technology Guidelines for PARCC 
Assessments,’® user interfaces were compatible with a range of common computer devices, operating 
systems, and browsers. 


Pearson also performed QC tests to verify that a standard set of responses was outputted to the XML as 
expected after PARCC had approved the final version of the form. These responses were based on the 
keys provided in the test map or a standard open-ended (OE) responses string that contained a valid 
range of characters. The test maps also were validated against the form layout and item types for 
correctness as part of these tests. 


Pearson conducted a multifaceted validation of all item layout, rendering, and functionality. Reviewers 
conducted comparisons between the approved item and the item as it appeared in the field-test form, 
validated that tools and functions in the test delivery system, TestNav, were accurately applied, and 
verified that the style and layout met requirements documented in the PARCC Publishing Style Guide as 
part of the PARCC Item Development Technical Guide.”° In addition, all answer keys were validated 
through a formal key review process. More details on the test development procedures are provided in 
Section 2. 


14.3 Quality Control of Test Materials 


Pearson provided high quality materials in a timely and efficient manner to meet PARCC’s test 
administration needs. Since the majority of printing work was done in-house, it was possible to fully 
control the production environment, press schedule, and quality process for print materials. 
Additionally, strict security requirements were employed to protect secure materials production; 
Section 3 provides details on the secure handling of test materials. Materials were produced according 
to the PARCC style guide and to the detailed specifications supplied in the materials list. 


'° This document is available online from: http://avocet.pearson.com/PARCC/Home#10429 
20 The PARCC Item Development Technical Guide is available online from: 
https://parcc-assessment.org/ 
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Pearson Print Service operates within the sanctions of an ISO 9001:2008 Quality Management System, 
and practices process improvement through Lean principles and employee involvement. 


Raw materials (paper and ink) used for scannable forms production were manufactured exclusively for 
Pearson Print Service using specifications created by Pearson Print Service. Samples of ink and paper 
were tested by Pearson prior to use in production. Project specialists were the point of contact for 
incoming production. 


Purchase orders and other order information were assessed against manufacturing capabilities and 
assigned to the optimal production methodology. PARCC expectations, quality requirements, and cost 
considerations were foremost in these decisions. Prior to release for manufacture, order information 
was checked against PARCC specifications, technical requirements, and other communication that 
includes expected outcomes. Records of these checks were maintained. 


Files for image creation flow through one of two file preparation functions: digital pre-press (DPP) for 
digital print methodology, or plateroom for offset print methodology. Both the DPP and plateroom 
functions verify content, file naming, imposition, pagination, numbering stream, registration of technical 
components, color mapping, workflow, and file integrity. Records of these checks are created and saved. 


Offset production requires printing that uses a lithographic process. Offline finishing activities are 
required to create books and package offset output. Digital output may flow through an inkjet digital 
production line (DPL) or a sheet-fed toner application process in the Xpress Center. A battery of quality 
checks was performed in these areas. The checks included color match, correct file selection, content 
match to proof, litho-code to serial number synchronization, registration of technical components, ink 
density controlled by densitometry, inspection for print flaws, perforations, punching, pagination, 
scanning requirements, and any unique features specified for the order. Records of these checks and 
samples pulled from planned production points were maintained. Offline finishing included cutting, 
shrink-wrapping, folding, and collating. The collation process has three robust inline detection systems 
that inspected each book for: 


e Caliper validation that detects too few or too many pages. This detector will stop the collator if 
an incorrect caliper reading is registered. 

e Anoptical reader that will only accept one sheet. Two or zero sheets will result in a collator 
stoppage. 

e The correct bar code for the signature being assembled. An incorrect or upside down signature 
will be rejected by the bar code scanner and will result in a collator stoppage. 


Pearson’s Quality Assurance (QA) department personnel inspected print output prior to collation and 
shipment. QA also supported process improvement, work area documentation, audited process 
adherence, and established training programs for employees. 


14.4 Quality Control of Scanning 
Establishing and maintaining the accuracy of scanning, editing, and imaging processes is a cornerstone 


of the Pearson scoring process. While the scanners are designed to perform with great precision, 
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Pearson implements other quality assurance processes to confirm that the data captured from scan 
processing produce a complete and accurate map to the expected results. 


Pearson pioneered optical mark reading (OMR) and image scanning, and continues to improve in-house 
scanners for this purpose. Software programs drive the capture of student demographic data and 
student responses from the test materials during scan processing. Routinely scheduled maintenance and 
adjustments to the scanner components (e.g., camera) maintain scanner calibration. Test sheets 
inserted into every batch test scanner accuracy and calibration. 


Controlled processes for developing and testing software specifications included a series of validation 
and verification procedures to confirm the captured data can be mapped accurately and completely to 
the expected results and that editing application rules are properly applied. 


14.5 Quality Control of Image Editing 


The final step in producing accurate data for scoring is the editing process. Once information from the 
documents was captured in the scanning process, the scan program file was executed, comparing the 
data captured from the student documents to the project specifications. The result of the comparison 
was a report (or edit listing) of documents needing corrections or validation. Image Editing Services 
performed the tasks necessary to correct and verify the student data prior to scoring. 


Using the report, editors verified that all unscanned documents were scanned, or the data were 
imported into the system through some other method such as flatbed scan or key entry. 


Documents with missing or suspect data were pulled, verified, and corrections or additional data were 
entered. Standard edits included: 


e Incorrect or double gridding 

e Incorrect dates (including birth year) 

e Mismatches between pre-ID label and gridded information 
e Incomplete names 


When all edits were resolved, corrections were incorporated into the document file containing student 
records. 


Additional quality checks were also performed. These included student N-count checks to make certain: 
e students were placed under the correct header, 
e all sheets belonged to the appropriate document, 
e documents were not scanned twice, and 
e no blank documents existed. 


Finally, accuracy checks were performed by checking random documents against scanned data to verify 
the accuracy of the scanning process. 


Once all corrections were made, the scan program was tested a second time to verify all data were valid. 
When the resulting output showed that no fields were flagged as suspect, the file was considered clean 
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and scoring began. Once all scanning was completed, the right/wrong response data were securely 
handed off. 


14.6 Quality Control of Answer Document Processing and Scoring 


Quality control of answer document processing and scoring involves all aspects of the scoring 
procedures, including key-based and rule-based machine scoring and hand-scoring for constructed- 
response items and performance tasks. 


For the 2015 PARCC operational administration, Pearson’s validation team prepared test plans used 
throughout the scoring process. Test plan preparation was organized around detailed specifications. 


Based on lessons learned from previous administrations, the following quality steps were implemented: 

e Raw score validation (e.g., score key validation; evidence statement, field-test non-score; 
double-grid combinations; possible correct combination, if applicable; out-of-range/negative 
test cases) 

e Matching (e.g., validation of high-confidence criteria, low-confidence criteria, cross document, 
external or forced matching by customer; prior to and after data updates; extract file of 
matched and unmatched documents) 

e Demographic update tests (e.g., verification of data extract against corresponding layout; valid 
values for updatable fields; invalid values for updatable/non-updatable fields; negative test for 
non-existing record or empty file) 


The following components were added to the quality control process specifically for the PARCC 
program. These additional steps were introduced to address issues with item-level scoring that were 
identified in the 2014 PARCC field-test administration: 


e XML Validation: A combination of automated validation against 100 percent of item XMLs and 
human inspection of XML from selected difficult item types or composite items. 

e = Administration/End-to-End Data Validation: An automated generation of response data from 
approved test maps that have known conditions against the operational scoring systems and 
data generation systems to verify scoring accuracy. 

e Psychometric Validation: Verification of data integrity using criteria typically used in 
psychometric processes (e.g., statistical keychecks) and categorization of identified issues to 
help inform investigation by other groups. 

e Content Validation: An examination, by subject matter experts, of all items using a combination 
of automated tools to generate response and scoring data. 


In addition to the steps described above, the following quality control process for answer keys and 
scoring that was implemented for the first PARCC operational administration was used: 


1. Pearson’s psychometrics team conducted empirical analyses based on preliminary data files and 
flagged items based on statistical criteria; 

2. Pearson content team reviewed the flagged items and provided feedback on the accuracy of 
content, answer keys, and scoring; 

3. Items potentially requiring changes were added to the product validation (PV) log for further 
investigation by other Pearson teams; 
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4. PARCC staff was notified of items for which keys or scoring changes were recommended; 
PARCC approved/rejected scoring changes; and 
6. All approved scoring changes were implemented and validated prior to the generation of the 


Gl 


data files used for psychometric processing. 
14.7 Quality Control of Psychometric Processes 


High quality psychometric work for the 2016-2017 PARCC operational administrations was necessary to 
provide accurate and reliable results of student performance. Pearson, HumRRO, and Measured 
Progress implemented quality control procedures to ensure the quality of the work including: 


Well-defined psychometric specifications 
Consistently applied data cleaning rules 
Clear and frequent communication 

Test run analyses 

Quality checks of the analyses 


Ow Pp we NP 


Checklists for statistical procedures 


14.7.1 Pearson Psychometric Quality Control Process 


Pearson was responsible for the psychometric analyses of the 2016-2017 PARCC operational 
administration and implemented measures to ensure the quality of work. The psychometric analyses 
were all conducted according to well-defined specifications. Data cleaning rules were clearly articulated 
and applied consistently throughout the process. Results from all analyses underwent comprehensive 
quality checks by a team of psychometricians and data analysts. Detailed checklists were used by 
members of the team for each statistical procedure. 


Described below is an overview of the quality control steps performed at different stages of the 
psychometric analyses. Greater detail is provided in Sections 6 (Classical Item Analysis), 7 (Differential 
Item Functioning), 10 (IRT Calibration and Scaling), and 12 (Scale Scores). 


Data Screening 

Data screening is an important first step to ensure quality data input for meaningful analysis. The 
Pearson Customer Data Quality (CDQ) team validated all student data files used in the operational 
psychometric analyses. The data validation for the student data files (SDF) and item response files (IRF) 
included the following steps: 


1. Validated variables in the data file for values in acceptable ranges. 

2. Validated that the test form ID, unique item numbers (UINs), and item sequence on the data file 
were consistent with the test form values on the corresponding test map. 

3. Computed the composite raw score, claim raw scores, and subclaim raw scores, given the item 
scores in the student data file. 

4. Compared computed raw scores to the raw scores in the student data file. 

Compared the student item response block (SIRB) to the item scores. 

6. Flagged student records with inconsistencies for further investigation. 


UI 
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Pearson Psychometrics and HumRRO established predefined valid case criteria, which were 
implemented consistently throughout the process. Refer to Section 5.3 for rules for inclusion of students 
in analyses and Section 10.2 for IRT calibration data preparation criteria and procedures. 


Classical Item Analysis 

Classical item analysis (IA) produces item level statistics (e.g., item difficulty and item-total correlations). 
The IA results were reviewed by Pearson psychometricians. Items flagged for unusual statistical 
properties were reviewed by the content team. A subset of items identified as having key issues, scoring 
issues, or content issues was presented to the PARCC Priority Alert Task Force, which made decisions on 
whether to exclude them from the IRT calibration process and, consequently, the calculation of reported 
student scores. Refer to Section 6.4 for classical IA item flagging criteria. 


Calibrations 

Creation of item response theory (IRT) sparse data matrices is an important step before the calibrations 
can begin. Using the same scored item response data, Pearson and HumRRO teams filtered the data and 
generated their own sparse data matrices independently. Processing of all data was done in parallel by 
two psychometricians and compared for n-counts. This verification of the data preparation was 
important to ensure that student exclusion rules were applied consistently across the analyses. 


During the calibration process, checks were made to ensure that the correct options for the analyses 
were selected. Checks were also made on the number of items, number of test takers with valid scores, 
IRT item difficulties, standard errors for the item difficulties, and the consistency between selected IRT 
statistics to the corresponding statistics obtained during item analyses. Psychometricians also 
performed detailed reviews of statistics to investigate the extent to which the assumptions of the model 
fit the observed data. Refer to Section 10.4 for IRT model fit evaluation criteria. 


Scaling 

During the scaling process, checks were made on the number of linking items, the number of items that 
were excluded from linking during the stability check of the scaling process, and the scaling constants. 
Linking items that did not meet the anchor criteria were excluded as linking items. For example, C-DIF 
items flagged in the mode comparability study were dropped. Additionally, items with large weighted 
root mean square difference (WRMSD) values in Round 1 of scaling were excluded as linking items in 
Round 2. Finally, reviewers computed the linking constants and then checked that the linking constants 
were correctly applied. Refer to Section 10.6 for description of scaling process. 


Conversion Tables 

Conversion tables must be accurate because they are used to generate reported scores for test takers. 
Comprehensive records were meticulously maintained on item-level decisions, and thorough checks 
were made to ensure that the correct items were included in the final score. All conversion tables were 
processed in parallel by Pearson and HumRRO and completely matched. A reasonableness check was 
also conducted by psychometricians for each content and grade level to make sure the results were in 
alignment with observations during the analyses prior to conversion table creation. Refer to Section 12.3 
for the procedure to create conversion tables. 
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Delivering Item Statistics 

Item statistics based on classical item analyses and IRT analyses were obtained during the psychometric 
analysis process. The statistics were compiled by two data analysts independently to ensure that the 
correct statistics were delivered for the item bank. 


14.7.2 HumRRO Psychometric Quality Control Process 


HumRRO served as the psychometric replicator for the 2016-2017 PARCC operational administration. 
HumRRO replicated the IRT analyses, scaling analyses, and the conversion file creations. The following 
steps outline the replication process: 


Calibrated online and paper data separately. 

Scaled the paper item parameter estimates to the online scale. 

Sent the item parameter estimates and scaling constants to Measured Progress for comparison. 
Reconciled differences, if any, in results with Pearson and Measured Progress. 

Sent data files to Measured Progress for comparison and reconciled differences, if any. 
Generated the performance levels, summative, claim, and subclaim conversion tables. 


pO) SO os te 1s > 


Sent conversion tables to Measured Progress for comparison and reconciled differences, if any. 
14.7.3 Measured Progress Psychometric Quality Control Process 


Measured Progress (MP) served as the external evaluator for the 2016-2017 PARCC operational 
administration. MP’s main task was to evaluate the reasonableness of IRT calibration results, and to 
compare to the IRT calibration results, scaling constants, summative, claim, and subclaim conversion 
tables created by HumRRO and Pearson. 


IRT Calibrations Comparison 
MP reviewed and compared the psychometric IRT calibrations performed primarily by Pearson and 
HumRRO for all grade levels in ELA/L and mathematics administered both online and on paper. 


Pearson and HumRRO each provided comparison files containing IRT item parameter estimates, IRT 
model fit statistics, and classical item statistics (item-level mean score, item-total correlation). Pearson 
also provided the IRT model fit plots for the items. For each test, the reasonableness of IRT parameters 
and the comparability of IRT parameter estimates between Pearson and HumRRO were evaluated on 
the following aspects: 


e Number of items and types of interventions in the IRT calibration process 

e Descriptive statistics of the IRT a-, b-, and d-parameter estimates 

e Scatterplot of IRT a-, b-, and d-parameter estimates 

e Absolute differences in IRT a-, b-, and d-parameter estimates 

e Mean absolute difference (MAD) and root mean square difference (RMSD) in IRT-model- 
predicted item mean scores if there were nontrivial absolute differences in IRT parameter 
estimates 
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e IRT model fit statistics and plots 


e |tem parameter linking status for paper forms 


IRT Comparison Results 

In general, MP observed highly comparable IRT item parameter estimates between Pearson and 
HumRRO across all grades and subjects and in both online and paper forms in the 2016-2017 PARCC 
operational analyses. The largest differences in item parameter estimates occurred at the fourth 
decimal place. In general, model fit ranged from good to reasonable, with a few items illustrating more 
variability when sample sizes were small. For a very few items across all the tests, MP observed extreme 
IRT parameter estimates and/or standard errors, and sent the findings to Pearson for further 
investigation. Those items were sent to the Priority Alert Task Force for review, and they were either 
spoiled in operational scoring or flagged for re-field-testing with larger sample sizes. 


Conversion Files Comparison 
MP provided comparison results for the scaling constants, the summative, claim, and subclaim score 
conversion tables and their performance levels of both regular and accommodated forms in each of the 
grades 3 through 8 and high school tests. 
The conversion tables were evaluated and compared in the following aspects: 

e Form ID and the number of total score points 

e Minimum and maximum score points 

e Raw cut-scores 

e Theta and scaled scores associated with each raw score point 


Conversion Files Comparison Results 

MP observed identical lower and upper limits of scale scores for summative and subclaim performance 
levels in each test. In the final comparison files and after any observed differences were reconciled, MP 
observed identical form IDs and number of raw score points for each test. The largest differences in the 
theta points only showed on the fourth decimals across all the conversion tables. Only very few scale 
score points differed by one point due to rounding out of the numerous forms across the grades and 
subjects. None of the differences occurred on the scale cut-scores. Overall, the final conversion files 
provided by HumRRO and Pearson were highly comparable. 
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Appendix 5: Test Takers by Grade and Mode, for Each State 


Table A.5.1 ELA/L Test Takers, by State, and Grade 


State 
PARCC 
PARCC 
PARCC 
PARCC 
PARCC 
BIE 

BIE 

BIE 

BIE 

BIE 

BIE 

co 

co 

co 

co 

co 

co 

DC 

DC 

DC 

DC 

DC 

DC 


Category 

N of Students 

N of CBT 

% of CBT 

N of PBT 

% of PBT 

N of Students 

% of PARCC Data 
N of CBT 

% of CBT 

N of PBT 

% of PBT 

N of Students 

% of PARCC Data 
N of CBT 

% of CBT 

N of PBT 

% of PBT 

N of Students 

% of PARCC Data 
N of CBT 

% of CBT 

N of PBT 

% of PBT 
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Total 
2,938,091 
2,764,202 

94.1 
173,889 
5.9 
8,762 
0.3 
2,000 
22.8 
6,762 
77.2 
416,423 
14.2 
394,006 
94.6 
22417 
5.4 
39,012 
1.3 
38,787 
99.4 
225 

0.6 


Grade 3 
420,496 
363,304 

86.4 
57,192 
13.6 
1,493 
0.1 
275 
18.4 
1,218 
81.6 
63,505 
2.2 
58,352 
91.9 
5153 
8.1 
6,349 
0.2 
6,288 
99.0 
61 

1.0 


Grade 4 
423,098 
374,192 

88.4 
48,906 
11.6 
1,445 
0.0 
277 
19.2 
1,168 
80.8 
64,065 
2.2 
59,938 
93.6 
4127 
6.4 
6,062 
0.2 
5,997 
98.9 
65 

1.1 


English Language Arts/Literacy 
Grade 7 Grade 8 


Grade 5 
414,461 
399,511 

96.4 
14,950 
3.6 
1,398 
0.0 
267 
19.1 
1,131 
80.9 
63,350 
2.2 
59,882 
94.5 
3468 
5.5 
5,385 
0.2 
5,340 
99.2 
45 

0.8 


Grade 6 
406,705 
389,472 

95.8 
17,233 
4.2 
1,301 
0.0 
241 
18.5 
1,060 
81.5 
60,776 
2.1 
57,781 
95.1 
2995 
4.9 
4,800 
0.2 
4,777 
99.5 
23 

0.5 


406,201 
390,667 
96.2 
15,534 
3.8 
1,087 
0.0 
316 
29.1 
771 
70.9 
58,706 
2.0 
56,140 
95.6 
2566 
4.4 
4,365 
0.1 
4,354 
99.7 
11 

0.3 
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399,371 
385,257 
96.5 
14,114 
3.5 
1,080 
0.0 
339 
31.4 
741 
68.6 
56,118 
1.9 
53,956 
96.1 
2162 
3.9 
4,290 
0.1 
4,282 
99.8 

8 

0.2 


Grade 9 
188,201 
185,251 

98.4 
2,950 
1.6 
175 
0.0 

36 
20.6 
139 
79.4 
49,903 
1.7 
47,957 
96.1 
1946 
3.9 
3,425 
0.1 


Grade 10 
171,965 
170,101 

98.9 
1,864 
1.1 
196 
0.0 
39 
19.9 
157 
80.1 
n/a 
n/a 
n/a 
n/a 
n/a 
n/a 
4,273 
0.1 


Grade 11 
107,593 
106,447 

98.9 
1,146 
1.1 
587 
0.0 
210 
35.8 
377 
64.2 
n/a 
n/a 
n/a 
n/a 
n/a 
n/a 
63 
0.0 
63 
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State Category Total Grade3 Grade4 Grade5 Grade6 Grade7 Grade8 Grade9 Grade10 Grade 11 
IL % of PARCC Data 29.6 5.0 5.0 4.9 4.9 5.0 4.9 n/a n/a n/a 
IL N of CBT 757,086 105,259 107,125 134,849 136,006 138,936 134,911 n/a n/a n/a 
IL % of CBT 86.9 71.9 72.6 93.2 94.2 95.5 94.6 n/a n/a n/a 
IL N of PBT 113,905 41,106 40,419 9,796 8,378 6,505 7,701 n/a n/a n/a 
IL % of PBT 13.1 28.1 27.4 6.8 5.8 4.5 5.4 n/a n/a n/a 
MD N of Students 467,310 68,754 68,080 66,143 63,391 63,335 62,810 1,907 56,882 16,008 
MD % of PARCC Data 15.9 2.3 2.3 2.3 2.2 2.2 2.1 0.1 1.9 0.5 
MD N of CBT 442,577 60,174 66,129 66,043 58,941 58,072 59,796 1,873 55,784 15,765 
MD % of CBT 94.7 87.5 97.1 99.8 93.0 91.7 95.2 98.2 98.1 98.5 
MD N of PBT 24,733 8,580 1,951 100 4,450 5,263 3,014 34 1,098 243 
MD % of PBT 5.3 12.5 2.9 0.2 7.0 8.3 4.8 1.8 1.9 1.5 
NJ N of Students 847,529 98,793 100,248 98,416 97,313 98,834 98,712 98,239 87,388 69,586 
NJ % of PARCC Data 28.8 3.4 3.4 3.3 3.3 3.4 3.4 3.3 3.0 2.4 
NJ N of CBT 845,081 98,615 100,030 98,207 97,161 98,654 98,511 97,714 86,967 69,222 
NJ % of CBT 99.7 99.8 99.8 99.8 99.8 99.8 99.8 99.5 99.5 99.5 
NJ N of PBT 2,448 178 218 209 152 180 201 525 421 364 
NJ % of PBT 0.3 0.2 0.2 0.2 0.2 0.2 0.2 0.5 0.5 0.5 
NM N of Students 214,366 24,701 24,885 24,580 24,359 23,852 23,437 24,096 23,107 21,349 
NM % of PARCC Data 7.3 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.8 0.7 
NM N of CBT 212,864 24,566 24,756 24,454 24,226 23,657 23,230 23,863 22,925 21,187 
NM % of CBT 99.3 99.5 99.5 99.5 99.5 99.2 99.1 99.0 99.2 99.2 
NM N of PBT 1,502 135 129 126 133 195 207 233 182 162 
NM % of PBT 0.7 0.5 0.5 0.5 0.5 0.8 0.9 1.0 0.8 0.8 
RI N of Students 73,517 10,529 10,764 10,534 10,375 10,574 10,304 410,437 n/a n/a 
Ri % of PARCC Data 2.5 0.4 0.4 0.4 0.4 0.4 0.4 0.4 n/a n/a 
RI N of CBT 71,620 9,768 9,935 10,459 10,333 10,531 10,224 10,370 n/a n/a 
RI % of CBT 97.4 92.8 92.3 99.3 99.6 99.6 99.2 99.4 n/a n/a 
RI N of PBT 1,897 761 829 75 42 43 80 67 n/a n/a 
Ri % of PBT 2.6 7.2 7.7 0.7 0.4 0.4 0.8 0.6 n/a n/a 


Note: CBT = computer-based test; PBT = paper-based test; n/a = not applicable. 
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State Category Total Grade3 Grade4 Grade5 Grade6 Grade7 Grade8 Al GO A2 M1 M2 M3 

PARCC_ N of Students 2,920,820 425,744 426,609 416,556 408,664 393,942 314,484 260,806 137,530 122,710 10964 2243 568 
PARCC Nof CBT 2,742,041 367,726 376,664 400,631 391,131 379,753 302,002 254,419 136,043 120,173 10892 2047 560 
PARCC %of CBT 93.9 86.4 88.3 96.2 95.7 96.4 96.0 97.6 98.9 97.9 99.3 91.3 98.6 
PARCC Nof PBT 178,779 58,018 49,945 15,925 17,533 14,189 12,482 6,387 1,487 2,537 72 196 8 
PARCC %of PBT 6.1 13.6 11.7 3.8 4.3 3.6 4.0 2.4 1.1 2.1 0.7 8.7 1.4 
BIE N of Students 8,749 1,489 1,441 1,399 1,299 1,081 1,065 175 199 375 21 +198 7 
BIE % of PARCC Data 0.3 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 
BIE N of CBT 1,990 274 274 268 239 310 325 51 71 170 2 6 0 
BIE % of CBT 22.7 18.4 19.0 19.2 18.4 28.7 30.5 29.1 35.7 45.3 9.5 3.0 0.0 
BIE N of PBT 6,759 1,215 1,167 1,131 1,060 771 740 124 128 205 19 192 7 
BIE % of PBT 77.3 81.6 81.0 80.8 81.6 71.3 69.5 70.9 64.3 54.7 90.5 97.0 100 
co N of Students 419,002 65,313 64,910 63,391 60,856 56,101 43,011 40,763 10,693 2,278 9,875 1,621 190 
co % of PARCC Data 14.3 2:2 2.2 2.2 2.1 1.9 1.5 1.4 0.4 0.1 0.3 0.1 0.0 
co N of CBT 388,447 59,430 59,941 58,931 56,351 52,038 39,992 37,832 10,068 2,192 9,861 1,621 190 
co % of CBT 92.7 91.0 92.3 93.0 92.6 92.8 93.0 92.8 94.2 96.2 99.9 100.0 100.0 
co N of PBT 30555 5883 4969 4460 4505 4063 3019 2931 625 86 14 0 0 
co % of PBT 7.3 9.0 7.7 7.0 7.4 7.2 7.0 7.2 5.8 3.8 0.1 0.0 0.0 
DC N of Students 38,684 6,410 6,102 5,408 4,862 4,280 3,609 3,288 4,319 291 n/a 115 n/a 
DC % of PARCC Data 1.3 0.2 0.2 0.2 0.2 0.1 0.1 0.1 0.1 0.0 n/a 0.0 n/a 
DC N of CBT 38,448 6,354 6,036 5,364 4,837 4,255 3,599 3,282 4,315 291 n/a 115 n/a 
DC % of CBT 99.4 99.1 98.9 99.2 99.5 99.4 99.7 99.8 99.9 100 n/a 100 n/a 
DC N of PBT 236 56 66 44 25 25 10 6 4 0 n/a 0 n/a 
DC % of PBT 0.6 0.9 1.1 0.8 0.5 0.6 0.3 0.2 0.1 0.0 n/a 0.0 n/a 
IL N of Students 872,611 146,790 147,821 144,901 144,642 144,794 138,847 4,293 409 114 n/a n/a n/a 
IL % of PARCC Data 29.9 5.0 5.1 5.0 5.0 5.0 4.8 0.1 0.0 0.0 n/a n/a n/a 
IL N of CBT 758,419 105,459 107,269 135,124 136,268 138,320 131,244 4,212 409 114 n/a n/a n/a 
IL % of CBT 86.9 71.8 72.6 93.3 94.2 95.5 94.5 98.1 100 100 n/a n/a n/a 
IL N of PBT 114,192 41,331 40,552 9,777 8,374 6,474 7,603 81 0 0 n/a n/a n/a 
IL % of PBT 13.1 28.2 27.4 6.7 5.8 4.5 5.5 1.9 0.0 0.0 n/a n/a n/a 
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MD N of Students 465,290 69,328 68,554 66,593 63,800 57,201 37,819 65,853 10,202 25,940 n/a n/a n/a 
MD % of PARCC Data 15.9 2.4 2.3 2.3 2.2 2.0 1.3 2.3 0.3 0.9 n/a n/a n/a 
MD N of CBT 444,055 60,908 66,556 66,492 60,564 54,764 37,142 63,400 10,078 24,151 n/a n/a n/a 
MD % of CBT 95.4 87.9 97.1 99.8 94.9 95.7 98.2 96.3 98.8 93.1 n/a n/a n/a 
MD N of PBT 21,235 8,420 1,998 101 3,236 2,437 677 2,453 124 1,789 n/a n/a n/a 
MD % of PBT 4.6 12.1 2.9 0.2 5.1 4.3 1.8 3.7 1.2 6.9 n/a n/a n/a 
NJ N of Students 828,775 99,913 101,271 99,367 98,235 95,960 63,595 109,499 87,151 73,779 n/a 5 Na 
NJ % of PARCC Data 28.4 3.4 3.5 3.4 3.4 3.3 2.2 3.7 3.0 2.5 n/a 0.0 n/a 
NJ N of CBT 826,338 99,718 101,044 99,152 98,077 95,776 63,407 108,967 86,732 73,460 n/a 5 n/a 
NJ % of CBT 99.7 99.8 99.8 99.8 99.8 99.8 99.7 99.5 99.5 99.6 n/a 100 n/a 
NJ N of PBT 2,437 195 227 215 158 184 188 532 419 319 n/a 0 n/a 
NJ % of PBT 0.3 0.2 0.2 0.2 0.2 0.2 0.3 0.5 0.5 0.4 n/a 0.0 n/a 
NM N of Students 213,072 25,824 25,627 24856 24505 23,835 19,093 26,100 21,815 19,708 1,049 289 371 
NM % of PARCC Data 7.3 0.9 0.9 0.9 0.8 0.8 0.7 0.9 0.7 0.7 0.0 0.0 0.0 
NM N of CBT 211,638 25,684 25,496 24,728 24,374 23,644 18,931 25,913 21,633 19,570 1,010 285 370 
NM % of CBT 99.3 99.5 99.5 99.5 99.5 99.2 99.2 99.3 99.2 99.3 96.3 98.6 99.7 
NM N of PBT 1,434 140 131 128 131 191 162 187 182 138 39 4 1 
NM % of PBT 0.7 0.5 0.5 0.5 0.5 0.8 0.8 0.7 0.8 0.7 3.7 1.4 0.3 
Ri N of Students 74,456 10,670 10,878 10,631 10,459 10,683 7,437. 10,731 2,742 225 n/a n/a n/a 
Ri % of PARCC Data 2.5 0.4 0.4 0.4 0.4 0.4 0.3 0.4 0.1 0.0 n/a n/a n/a 
RI N of CBT 72,525 9,892 10,043 10,562 10,415 10,639 7,354 10,658 2,737 225 n/a n/a n/a 
RI % of CBT 97.4 92.7 92.3 99.4 99.6 99.6 98.9 99.3 99.8 100.0 n/a n/a n/a 
Ri N of PBT 1,931 778 835 69 44 44 83 73 5 0 n/a n/a n/a 
Ri % of PBT 2.6 7.3 7.7 0.6 0.4 0.4 1.1 0.7 0.2 0.0 n/a n/a n/a 


Note: Ail = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated Mathematics II], M3 = Integrated Mathematics III. CBT = 


computer-based test; PBT = paper-based test; n/a = not applicable. 
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State’ Category Total Grade3 Grade4 Grade5 Grade6 Grade7 Grade8& Al GO A2 

PARCC  N of Students 28,279 6,399 3,971 3,021 2,736 2,617 2,573 4,032 1,755 1,085 
PARCC Nof CBT 26,347 5,498 3,532 2,944 2,694 2,572 2,532 3,646 1,755 1,084 
PARCC %of CBT 93.2 85.9 88.9 97.5 98.5 98.3 98.4 90.4 100 99.9 
PARCC’ Nof PBT 1,932 901 439 77 42 45 41 386 n/a 1 
PARCC %of PBT 6.8 14.1 11.1 2.5 1.5 1.7 1.6 9.6 n/a 0.1 
BIE N of Students n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 
BIE N of CBT n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 
BIE % of CBT n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 
BIE N of PBT n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 
BIE % of PBT n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 
co N of Students 3,373 1,413 718 319 219 225 261 161 3 n/a 
co % of PARCC Data 11.9 5.0 2.5 1.1 0.8 0.8 0.9 0.6 0.0 n/a 
co N of CBT 3,089 1,322 656 258 201 203 240 152 3 n/a 
co % of CBT 91.6 93.6 91.4 80.9 91.8 90.2 92.0 94.4 100 n/a 
co N of PBT 284 91 62 61 18 22 21 9 n/a n/a 
co % of PBT 8.4 6.4 8.6 19.1 8.2 9.8 8.0 5.6 n/a n/a 
DC N of Students 447 67 83 67 60 55 53 38 24 n/a 
DC % of PARCC Data 1.6 0.2 0.36 0.2 0.2 0.2 0.2 0.1 0.1 n/a 
DC N of CBT 446 67 83 67 60 54 53 38 24 n/a 
DC % of CBT 99.8 100 100 100 100 98.2 100 100 100 n/a 
DC N of PBT 1 n/a n/a n/a n/a 1 n/a n/a n/a n/a 
DC % of PBT 0.2 n/a n/a n/a n/a 1.8 n/a n/a n/a n/a 
IL N of Students 6,123 2,267 1,102 1,026 842 444 442 n/a n/a n/a 
IL % of PARCC Data 21.7 8.0 3.9 3.6 3.0 1.6 1.6 n/a n/a n/a 
IL N of CBT 4,943 1,481 741 1,014 831 439 437 n/a n/a n/a 
IL % of CBT 80.7 65.3 67.2 98.8 98.7 98.9 98.9 n/a n/a n/a 
IL N of PBT 1,180 786 361 12 11 5 5 n/a n/a n/a 
IL % of PBT 19.3 34.7 32.8 1.2 1.3 1.1 1.1 n/a n/a n/a 
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Mathematics (Spanish-Language forms) 


State’ Category Total Grade3 Grade4 Grade5 Grade6 Grade7 Grades Al GO A2 M1 M2 M3 
MD N of Students 825 26 53 59 57 51 74 497 n/a 8 n/a n/a n/a 
MD % of PARCC Data 2.9 0.1 0.2 0.2 0.2 0.2 0.3 1.8 n/a 0.0 n/a n/a n/a 
MD N of CBT 450 26 53 59 55 49 67 133 n/a 8 n/a n/a n/a 
MD % of CBT 54.5 100 100 100 96.5 96.1 90.5 26.8 n/a 100 n/a n/a n/a 
MD N of PBT 375 n/a n/a n/a 2 2 7 364 n/a n/a n/a n/a_ n/a 
MD % of PBT 45.5 n/a n/a n/a 3.5 3.9 9.5 73.2 n/a n/a n/a n/a n/a 
NJ N of Students 13,039 1,380 1,188 1,144 1,244 1,458 1,387 2,719 1,543 976 n/a n/a n/a 
NJ % of PARCC Data 46.1 4.9 4.2 4.0 4.4 5.2 4.9 9.6 5.5 3.55 n/a n/a n/a 
NJ N of CBT 12,967 1,363 1,177 1,140 1,235 1,445 1,383 2,706 1,543 975 n/a n/a n/a 
NJ % of CBT 99.4 98.8 99.1 99.7 99.3 99.1 99.7 99.5 100 99.9 n/a n/a n/a 
NJ N of PBT 72 17 11 4 9 13 4 13 n/a 1 n/a n/a n/a 
NJ % of PBT 0.6 1.2 0.9 0.3 0.7 0.9 0.3 0.5 n/a 0.1 n/a n/a “n/a 
NM N of Students 3,406 1,105 742 299 212 257 233 262 159 101 34 1 n/a 
NM % of PARCC Data 12.0 3.9 2.6 1.1 0.7 0.9 0.8 0.9 0.6 0.4 O11 O.0 n/a 
NM N of CBT 3,398 1,102 740 299 211 256 232 262 159 101 34 1 n/a 
NM % of CBT 99.8 99.7 99.7 100 99.5 99.6 99.6 100 100 100 100 100 n/a 
NM N of PBT 8 3 2 n/a 1 1 1 n/a n/a n/a n/a n/a _ n/a 
NM % of PBT 0.2 0.3 0.3 n/a 0.5 0.4 0.4 n/a n/a n/a n/a n/a n/a 
Ri N of Students 1,065 141 85 107 102 127 123 354 26 n/a n/a n/a _ n/a 
Ri % of PARCC Data 3.8 0.5 0.3 0.4 0.4 0.4 0.4 1.3 0.1 n/a n/a n/a. n/a 
Ri N of CBT 1,053 137 82 107 101 126 120 354 26 n/a n/a n/a _ n/a 
Ri % of CBT 98.9 97.2 96.5 100 99 99.2 97.6 100 100 n/a n/a n/a _ n/a 
Ri N of PBT 12 4 3 n/a 1 1 3 n/a n/a n/a n/a n/a. n/a 
Ri % of PBT 1.1 2.8 3.5 n/a 1.0 0.8 2.4 n/a n/a n/a n/a n/a. n/a 


Note: Ail = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated Mathematics II], M3 = Integrated Mathematics III. CBT = 
computer-based test; PBT = paper-based test; n/a = not applicable. 
“No students in BIE tested in mathematics using Spanish-language forms. 
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Gender 
Valid Cases 
Grade Mode Female Male 

N N % N % 

3 All 420,496 206,149 49.0 214,347 51.0 
3 CBT 363,304 177,828 48.9 185,476 51.1 
3 PBT 57,192 28,321 49.5 28,871 50.5 
4 All 423,098 207,393 49.0 215,705 51.0 
4 CBT 374,192 183,247 49.0 190,945 51.0 
4 PBT 48,906 24,146 49.4 24,760 50.6 
5 All 414,461 203,177 49.0 211,284 51.0 
5 CBT 399,511 196,067 49.1 203,444 50.9 
5 PBT 14,950 7,110 47.6 7,840 52.4 
6 All 406,705 198,893 48.9 207,812 51.1 
6 CBT 389,472 190,608 48.9 198,864 51.1 
6 PBT 17,233 8,285 48.1 8,948 51.9 
7 All 406,201 198,297 48.8 207,904 51.2 
7 CBT 390,667 190,734 48.8 199,933 51.2 
7 PBT 15,534 7,563 48.7 7,971 51.3 
8 All 399,371 194,852 48.8 204,519 51.2 
8 CBT 385,257 188,111 48.8 197,146 51.2 
8 PBT 14,114 6,741 47.8 7,373 52.2 
9 All 188,201 90,951 48.3 97,250 51.7 
9 CBT 185,251 89,626 48.4 95,625 51.6 
9 PBT 2,950 1,325 44.9 1,625 55.1 
10 All 171,965 89,918 48.8 88,047 51.2 
10 CBT 170,101 83,064 48.8 87,037 51.2 
10 PBT 1,864 854 45.8 1,010 54.2 
11 All 107,593 51,199 47.6 56,394 52.4 
11 CBT 106,447 50,716 47.6 55,731 52.4 
11 PBT 1,146 483 42.1 663 57.9 


Note: CBT = computer-based tests; PBT = paper-based tests. 
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Table A.5.5 All States Combined: All Mathematics Test Takers by Grade, Mode, and Gender 


Gender 
Valid Cases 
Grade Mode Female Male 

N N % N % 

3 All 425,744 208,739 49.0 217,005 51.0 
3 CBT 367,726 179,993 48.9 187,733 51.1 
3 PBT 58,018 28,746 49.5 29,272 50.5 
4 All 426,609 209,075 49.0 217,534 51.0 
4 CBT 376,664 184,410 49.0 192,254 51.0 
4 PBT 49,945 24,665 49.4 25,280 50.6 
5 All 416,556 204,220 49.0 212,336 51.0 
5 CBT 400,631 196,608 49.1 204,023 50.9 
5 PBT 15,925 7,612 47.8 8,313 52.2 
6 All 408,664 199,820 48.9 208,844 51.1 
6 CBT 391,131 191,410 48.9 1999,721 51.1 
6 PBT 17,533 8,410 48.0 9,123 52.0 
7 All 393,942 192,605 48.9 201,337 51.1 
7 CBT 379,753 185,734 48.9 194,019 51.1 
7 PBT 14,189 6,871 48.4 7,318 51.6 
8 All 314,484 151,133 48.1 163,351 51.9 
8 CBT 302,002 145,209 48.1 156,793 51.9 
8 PBT 12,482 5,924 47.5 6,558 52.5 
Al All 260,806 126,562 48.5 134,244 51.5 
Al CBT 254,419 123,595 48.6 130,824 51.4 
Al PBT 6,387 2,967 48.5 3,420 53.5 
GO All 137,530 67,407 49.0 70,123 51.0 
GO CBT 136,043 66,716 49.0 69,327 51.0 
GO PBT 1,487 691 46.5 796 53.5 
A2 All 122,710 61,281 49.9 61,429 50.1 
A2 CBT 120,173 60,114 50.0 60,059 50.0 
A2 PBT 2,537 1,167 46.0 1,370 54.0 
M1 All 10,964 5,095 46.5 5,869 53.5 
M1 CBT 10,892 5,063 46.5 5,829 53.5 
M1 PBT 72 32 44.4 40 55.6 
M2 All 2,243 1,102 49.1 1,141 50.9 
M2 CBT 2,047 1,005 49.1 1,042 50.9 
M2 PBT 196 97 49.5 99 50.5 
M3 All 568 274 48.2 294 51.8 
M3 CBT 560 272 48.6 288 51.4 
M3 PBT 8 2 25.0 6 75.0 


Note: Includes students taking English-language mathematics tests, students taking Spanish-language mathematics tests, 
and students taking accommodated forms. A1 = Algebra 1, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, 
M2 = Integrated Mathematics Il, M3 = Integrated Mathematics III. CBT = computer-based test; PBT = paper-based test; and 
n/a = not applicable. 
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Table A.5.6 All States Combined: Spanish-Language Mathematics Test Takers, by Grade, Mode, and Gender 


: Gender 
Grade Mode velldikaces Female Male 

N N % N % 

3 All 6,399 3,188 49.8 3,211 50.2 
3 CBT 5,498 2,733 49.7 2,765 50.3 
3 PBT 901 455 50.5 446 49.5 
4 All 3,971 1,879 47.3 2,092 52.7 
4 CBT 3,532 1,657 46.9 1,875 53.1 
4 PBT 439 222 50.6 217 49.4 
5 All 3,021 1,448 47.9 1,573 52.1 
5 CBT 2,944 1,405 47.7 1,539 52.3 
5 PBT 77 43 55.8 34 44.2 
6 All 2,736 1,288 47.1 1,448 52.9 
6 CBT 2,694 1,266 47.0 1,428 53.0 
6 PBT 42 22 52.4 20 47.6 
7 All 2,617 1,230 47.0 1,387 53.0 
7 CBT 2,572 1,217 47.3 1,355 52.7 
7 PBT 45 13 28.9 32 71.1 
8 All 2,573 1,230 47.8 1,343 52.2 
8 CBT 2,532 1,213 47.9 1,319 52.1 
8 PBT 41 17 41.5 24 58.5 
Al All 4,032 1,762 43.7 2,270 56.3 
Al CBT 3,646 1,596 43.8 2,050 56.2 
Al PBT 386 166 43.0 220 57.0 
GO All 1,755 834 47.5 921 52.5 
GO CBT 1,755 834 47.5 921 52.5 
GO PBT n/a n/a n/a n/a n/a 
A2 All 1,085 521 48.0 564 52.0 
A2 CBT 1,084 520 48.0 564 52.0 
A2 PBT 1 1 100 n/a n/a 
M1 All 88 41 46.6 47 53.4 
M1 CBT 88 41 46.6 47 53.4 
M1 PBT n/a n/a n/a n/a n/a 
M2 All n/a n/a n/a 1 100 
M2 CBT n/a n/a n/a 1 100 
M2 PBT n/a n/a n/a n/a n/a 
M3 All n/a 1 100 n/a n/a 
M3 CBT n/a 1. 100 n/a n/a 
M3 PBT n/a n/a n/a n/a n/a 


Note: Ail = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated Mathematics II], M3 = 
Integrated Mathematics III. CBT = computer-based test; PBT = paper-based test; and 
n/a = not applicable. 
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Table A.5.7 Demographic Information for Grade 3 ELA/L, Overall and by State 


Demneeraphie PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 49.8 99.1 44.9 79.3 54.7 45.1 40.1 73.2 51.8 
SWD (%) 14.7 123 #125 154 142 #4133 176 «49158 15.7 
EL (%) 14.5 33.4 15.1 10.4 21.1 11.3 6.5 17.0 11.7 
Male (%) 51.0 483 513 502 508 513 509 511 506 
Female (%) 49.0 51.7 487 498 492 487 4.491 489 49.4 
Amind/ANat (%) 1.4 99.5 0.7 n/r 0.7 0.4 0.1 10.2 07 
Asian (%) 5.9 n/r 3.1 14 49 6.4 10.6 1.0 3.5 
Black/AA (%) 17.4 n/r 4.8 68.0 176 340 153 21 «8.5 
Hisp/Lat (%) 28.3 n/r «=«33.32=S«O«17.0s 2650S 173~Ss—«83s—s«.—«éLB 
Wh/Caus (%) 35.1 n/r 0.3 104 463 369 426 235 555 
NtvHawaii/Pacific (%) 

8.2 n/r 53.3 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 3.6 n/r 4.6 2.0 3.6 4.9 a2 2.2 47 
Unknown (%) 0.2 n/r n/r 0.9 0.2 n/r 0.6 n/r 0.2 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two 
or more races reported; n/r = not reported. 
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Table A.5.8 Demographic Information for Grade 4 ELA/L, Overall and by State 


Deneeraphie PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 49.2 988 452 787 539 440 396 734 517 
SWD (%) 15.8 142 131 #173 «24152 4145 «190 164 158 
EL (%) 9.4 390 133 89 114 7.0 41 140 7.9 
Male (%) 51.0 509 513 505 508 512 509 511 511 
Female (%) 49.0 491 487 495 492 488 491 489 489 
AmInd/ANat (%) 1.4 99.0 07 n/r 0.6 0.3 01 103 06 
Asian (%) 5.7 n/r al 1.3 4.8 63 10.2 ae 
Black/AA (%) 17.0 n/r 47 681 1714 335 15.2 18 8.7 
Hisp/Lat (%) 28.6 n/r =o 344 6B SCOiDSsid17<3'—‘iKASC“‘éw CSCS 
Wh/Caus (%) 35.6 n/r 03 108 468 37.7 430 23.2 567 
NtvHawaii/Pacific (%) 

8.1 n/r 52.4 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 3.4 n/r 4.5 1.9 3.5 4.7 2.2 20 48 
Unknown (%) 0.2 n/r n/r 0.9 0.3 n/r 0.6 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported; n/a = not applicable. 
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Table A.5.9 Demographic Information for Grade 5 ELA/L, Overall and by State 


Denmoetaphie PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 48.3 99.6 45.1 77.9 53.0 42.8 38.4 72.7 49.2 
SWD (%) 16.3 15.3 137 4182 154 155 194 17.0 164 
EL (%) 7.0 49.2 12.7 7.7 7.2 4.7 2.8 12.3 5.7 
Male (%) 51.0 46.9 512 508 508 513 509 50.7 SLI 
Female (%) 49.0 53.1 488 49.2 49.2 487 4491 493 483 
Amind/ANat (%) 1.3 99.8 0.7 n/r 0.6 0.3 0.1 10.3 0.6 
Asian (%) 5.8 n/r 3.1 14 49 6.5 10.3 1.1 3.2 
Black/AA (%) 16.6 n/r 4.6 685 16.7 335 15.0 18 8.0 
Hisp/Lat (%) 28.3 nfr 345 17.5 «26.70 16.7) 27.365 5S 
Wh/Caus (%) 36.3 n/r 0.2 9.7 47.6 38.2 445 23.2 583 
NtvHawaii/Pacific (%) 

8.2 n/r 52.7 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 3.3 n/r 41 1.9 3.4 4.6 2.0 a4). ag 
Unknown (%) 0.2 n/r n/r 0.7 0.1 n/r 0.6 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported. 
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Table A.5.10 Demographic Information for Grade 6 ELA/L, Overall and by State 


Daneerap hie PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 47.2 995 448 749 518 423 372 £698 485 
SWD (%) 16.3 159 135 184 152 156 195 171 174 
EL (%) 5.9 464 106 5.9 6.1 41 24 102 5.2 
Male (%) 51.1 494 513 503 513 508 511 506 51.0 
Female (%) 48.9 506 487 498 487 492 489 494 49.0 
AmInd/ANat (%) 1.2 995 07 n/r 0.4 0.3 0.1 97 08 
Asian (%) 5.9 n/r 3.4 1.3 5.0 6.8 10.2 i2 29 
Black/AA (%) 16.4 n/r 46 709 163 336 147 #219 ~« 83 
Hisp/Lat (%) 28.1 n/r = 35.3155 265) 15.9 ©9268 = 6.92 
Wh/Caus (%) 37.2 n/r 0.3 98 483 39.0 457 235 593 
NtvHawaii/Pacific (%) 

7.8 n/r 51.7 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 44 n/r 4.0 1.9 3.2 4.4 1.8 18 44 
Unknown (%) 0.2 n/r n/r n/r 0.1 n/r 0.5 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported. 
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Table A.5.11 Demographic Information for Grade 7 ELA/L, Overall and by State 


Daneranhie PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 45.6 993 436 751 504 403 353 698 466 
SWD (%) 164 140 130 204 151 #2158 193 £49166 173 
EL (%) 5.7 394 107 62 5.5 41 24 114 49 
Male (%) 51.2 503 515 494 512 510 513 509 SLI 
Female (%) 48.8 497 485 506 488 49.0 487 491 48.3 
AmInd/ANat (%) 1.2 995 08 n/r 0.3 0.2 01 105 08 
Asian (%) 6.0 n/r 3.4 1.4 5.0 66 103 19 -32 
Black/AA (%) 16.3 n/r 45 709 161 338 147 42.20 ~°«83 
Hisp/Lat (%) 27.4 n/p =—o35.2—i«dSBCOHLia1—iSLT—s«CD—s«2.1 
Wh/Caus (%) 38.3 n/r 0.2 97 491 400 470 229 59.7 
NtvHawaii/Pacific (%) 

7.6 n/r 52.2 n/r 0.1 0.1 0.2 n/r 0.2 
Two or More (%) 3.0 n/r a7 1.6 3.2 4.4 1.6 i7- 3g 
Unknown (%) 0.2 n/r n/r n/r 0.1 n/r 0.4 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported. 
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Table A.5.12 Demographic Information for Grade 8 ELA/L, Overall and by State 


Deneeraphic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 44.9 99.7 431 750 500 391 343 £692 45.6 
SWD (%) 15.9 158 131 195 149 155 191 156 17.8 
EL (%) 5.7 414 110 #60 5.2 4.3 26 110 56 
Male (%) 51.2 496 515 503 515 510 511 504 51.3 
Female (%) 48.8 504 485 497 485 490 489 496 487 
AmInd/ANat (%) 1.1 996 0.7 n/r 0.3 0.2 01 100 06 
Asian (%) 6.0 n/r 3.4 1.9 4.9 67 103 12 27 
Black/AA (%) 16.5 n/r 46 715 162 334 151 #219 ~°« 83 
Hisp/Lat (%) 272 n/r = 35.7 «15.6 So 26.0s 15.0») 24.9) 62.3 23.9 
Wh/Caus (%) 38.8 n/r 0.3 90 493 401 478 228 61.0 
NtvHawaii/Pacific (%) 

7.4 n/r 51.7 n/r 0.1 0.1 0.2 n/r n/r 
Two or More (%) 2.8 n/r 3.6 1.5 3.0 4.5 1.3 i7 32 
Unknown (%) 0.1 n/r n/r n/r 0.1 n/r 0.4 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported; n/a = not applicable. 
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Table A.5.13 Demographic Information for Grade 9 ELA/L, Overall and by State 


Demoerplile PARCC BIE co DC IL MD NJ NM RI 
“EconDis(%) = += 44.6~=©—S 98.3. 42.2—SO794onfrsiAS si C« GLO 
SWD (%) 16.0 16.6 12.6 17.9 n/r 13.7 17.9 14.7 17.1 
EL (%) 8.3 33.7 12.2 8.4 n/r 3.0 5.6 12.0 6.1 
Male (%) 51.7 51.4 52.7 50.4 n/r 54.2 51.2 51.1 52.1 
Female (%) 48.3 48.6 47.3 49.6 n/r 45.8 48.8 48.9 47.9 
Amind/ANat (%) 1.8 100 0.8 n/r n/r n/r 0.1 11.1 0.7 
Asian (%) 6.5 n/r 3.5 1.5 n/r 2.0 9.9 1.2 3.1 
Black/AA (%) 112 n/r 5.1 69.9 ade 16.0 14.7 2.0 8.4 
Hisp/Lat (%) 33.1 n/r 364 171 alt 8.6 26.6 612 243 
Wh/Caus (%) 31.6 n/r 0.2 8.9 n/r 66.3 46.8 22.9 60.0 
NtvHawaii/Pacific (%) 13.5 n/r 50.4 n/r vr n/r 0.2 n/r n/r 
Two or More (%) 2.0 n/r 3.5 1.3 n/t 6.7 1.2 1.6 3.1 
Unknown (%) 0.3 n/r n/r 0.8 n/t n/r 0.5 n/r 0.2 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported due to n<20; n/a = not applicable. 
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Table A.5.14 Demographic Information for Grade 10 ELA/L, Overall and by State 


Demographic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 39.0 98.0 n/t 77.6 n/r 35.4 34.2 58.4 —n/r 
SWD (%) 16.6 14.3 n/r 19.6 n/r 150 182 134. ofr 
EL (%) 5.5 26.5 n/t 8.7 n/r 4.8 4.7 9.6 n/r 
Male (%) 51.2 46.9 n/r 50.1 nr = 50.65.7507 nr 
Female (%) 48.8 53.1 n/t 49.9 n/r 49.4 48.3 49.3 n/r 
Amind/ANat (%) iG 98.5 n/r n/r n/t 0.2 0.1 10.1 9 n/r 
Asian (%) 7.6 n/r n/r 1.3 n/r 7A 9.9 1.4 n/r 
Black/AA (%) 21.5 n/r n/t 71.3 n/r 35.2 15.3 1.9 n/r 
Hisp/Lat (%) 26.8 n/r n/r 16.4 n/r 143 266 609 = nfr 
Wh/Caus (%) 40.0 n/r n/r 6.4 n/r 39.1 46.4 241 ofr 
NtvHawaii/Pacific (%) 

0.1 n/r n/r n/r n/r 0.1 0.2 n/r n/r 
Two or More (%) 2.1 n/r n/r 2.5 n/r 3.9 1.0 1.6 n/r 
Unknown (%) 0.3 n/r n/r 2.0 n/r n/r 0.4 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported; n/a = not applicable. 
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Table A.5.15 Demographic Information for Grade 11 ELA/L, Overall and by State 


Danoeran hie PARCC BIE co DC IL MA MD NJ NM RI 
Econ Dis (%) 42.0 98.1 n/r 77.8 n/r 43.4 364 575 n/r 42.0 
SWD (%) 17.7 12.4 n/r 38.1 n/r 17.0 196 12.5 n/r 17.7 
EL (%) 4.9 20.3 n/r n/r n/r iby 4.7 pBe n/r 4.9 
Male (%) 52.4 52.0 n/r 60.3 n/r 53.3 «52.8 ~~ 50.5 n/r 52.4 
Female (%) 47.6 48.0 n/r 39.7 n/r 46.7 47.2 49.5 n/r 47.6 
Amind/ANat (%) 2.8 99.0 = nfr n/r n/r 0.3 0.1 105 nf/r 28 
Asian (%) 6.1 n/r n/r n/r n/r 1.8 8.5 1.3 n/r 6.1 
Black/AA (%) 19.9 n/r n/r n/r n/r 58.5 16.5 2.3 n/r 19.9 
Hisp/Lat (%) 32.0 n/r n/r = s65.1—ssanfr——(iid20.2s—‘i (iti. Sstiéir’—Ss 332.0 
Wh/Caus (%) 37.4 n/r n/r n/r n/r 26.4 444 24.1 n/r 37.4 
NtvHawaii/Pacific (%) 

0.2 n/r n/r n/r n/r 0.1 0.2 n/r n/r 0.2 
Two or More (%) 1.4 n/r n/r n/r n/r 2.6 1.1 1.6 n/r 1.4 
Unknown (%) 0.4 n/r n/r n/r n/r n/r 0.6 n/r n/r 0.4 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported due to n<20; n/a = not applicable. 
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Table A.5.16 Demographic Information for Grade 3 Mathematics, Overall and by State 


Dencerphic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 50.1 99.1 46.2 79.3 54.7 45.2 40.3 74.3 51.9 
SWD (%) 14.6 12.2 124 153 141 #429132 #174 «29156 15.6 
EL (%) 15.7 33.4 17.4 11.3 21.4 12.1 7.6 20.7. 12.9 
Male (%) 51.0 482 513 504 508 513 509 511 504 
Female (%) 49.0 518 487 496 492 487 4491 489 49.6 
Amind/ANat (%) 1.4 99.5 0.6 n/r 0.7 0.4 0.1 98 O07 
Asian (%) 5.9 n/r 3.0 1.5 5.0 6.5 10.6 1.0 3.6 
Black/AA (%) 17.2 n/r 4.6 673 176 338 15.2 2.0 85 
Hisp/Lat (%) 28.9 nfr 35.10 176 S265) 17.78.7625 Ss27.2 
Wh/Caus (%) 34.8 n/r 0.3 104 463 366 422 226 55.0 
NtvHawaii/Pacific (%) 

8.1 n/r 51.9 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 3.5 n/r 4.5 2.0 3.6 4.9 2,3 24 4.7 
Unknown (%) 0.2 n/r n/r 0.9 0.2 n/r 0.6 n/r 0.2 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported. 
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Table A.5.17 Demographic Information for Grade 4 Mathematics, Overall and by State 


Deneeraphic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 49.4 98.8 45.9 78.8 53.9 44.1 39.8 74.1 51.9 
SWD (%) 15.6 142 130 172 «249151 143 £4188 162 415.6 
EL (%) 10.3 39.1 14.5 9.6 11.8 7.8 5.2 16.6 9.0 
Male (%) 51.0 509 513 505 508 512 509 512 511 
Female (%) 49.0 491 487 495 492 488 491 488 48.9 
Amind/ANat (%) 1.3 99.0 0.7 n/r 0.6 0.3 0.1 10.0 06 
Asian (%) 5.8 n/r 3.0 1.4 49 6.4 10.3 11 3.2 
Black/AA (%) 16.8 n/r 4.6 676 17.0 33.3 15.1 18 88 
Hisp/Lat (%) 29.1 n/r =o 35.30 17.3269) 17.7288 62.7261 
Wh/Caus (%) 35.3 n/r 0.3 10.7 46.7 375 42.7 226 563 
NtvHawaii/Pacific (%) 

8.0 n/r 51.7 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 3.4 n/t 4.4 1.8 3.5 4.6 24 2.0 47 
Unknown (%) 0.3 n/r n/r 0.9 0.3 n/r 0.6 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported; n/a = not applicable. 
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Table A.5.18 Demographic Information for Grade 5 Mathematics, Overall and by State 


Deneeraphic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 48.3 99.6 45.1 78.0 53.1 43.0 38.6 72.9 49.3 
SWD (%) 16.2 15.3 137 44180 153 153 192 168 162 
EL (%) 7.6 49.2 19:8 8.4 7.5 5.5 3.8 13.4 6.7 
Male (%) 51.0 470 512 506 508 513 509 50.7 517 
Female (%) 49.0 53.0 488 494 492 487 491 493 483 
Amind/ANat (%) 1.3 99.8 0.7 n/r 0.6 0.3 0.1 10.1 06 
Asian (%) 5.9 n/r 3.1 1.5 4.9 6.5 10.3 1.1 3.3 
Black/AA (%) 16.5 n/r 4.6 679 166 333 149 18 8.0 
Hisp/Lat (%) 28.5 n/r =o 346 0—Cid8:1 26.7020 17Si27SCi—s«é8S'B 
Wh/Caus (%) 36.2 n/r 0.2 9.7 47.6 38.0 442 23.0 57.8 
NtvHawaii/Pacific (%) 

8.1 n/r 52.7 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 3.3 n/r 4.1 1.9 3.4 4.6 2.0 234 4.1 
Unknown (%) 0.2 n/r n/r 0.8 0.1 n/r 0.7 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported. 
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Table A.5.19 Demographic Information for Grade 6 Mathematics, Overall and by State 


Deneaaphic PARCC _BIE co DC IL MD NJ NM RI 
Econ Dis (%) 47.3 994 448 749 518 427 375 701 48.7 
SWD (%) 16.2 i614 135 180 154 185 494 168 172 
EL (%) 6.5 465 108 7.1 6.4 5.0 a5 wa  €3 
Male (%) iA 496 513 503 513 508 511 506 509 
Female (%) 48.9 504 487 497 487 492 489 494 491 
AmInd/ANat (%) i 994 O07 n/r 0.4 0.3 0.1 96 08 
Asian (%) 6.0 n/r 3.4 1.3 5.0 67 102 12 29 
Black/AA (%) 16.4 n/r 46 701 163 334 147 #219 ~ «84 
Hisp/Lat (%) 28.3 n/r = 35.3163 26.616 A272) 62.2247 
Wh/Caus (%) a7 n/r 0.3 99 482 387 453 232 588 
NtvHawaii/Pacific (%) 

7.8 n/r 51.7 n/r 0.1 0.2 0.2 n/r n/r 
Two or More (%) 3.1 n/r 4.0 1.9 3.2 4.3 1.8 18 «41 
Unknown (%) 0.2 n/r n/r n/r 0.1 n/r 0.5 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two 
or more races reported; n/r = not reported. 
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Table A.5.20 Demographic Information for Grade 7 Mathematics, Overall and by State 


Denieeraphic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 46.9 993 452 769 506 435 366 704 469 
SWD (%) 16.5 140 134 205 151 #2169 #198 165 17.2 
EL (%) 6.5 s65 aia 72 5.8 5.4 a7 We 60 
Male (%) Sid 503 512 495 512 508 51.2 508 516 
Female (%) 48.9 497 488 505 488 492 488 49.2 48.4 
AmInd/ANat (%) 1.2 995 08 n/r 0.3 0.2 01 104 09 
Asian (%) 5.4 n/r pl 1.3 5.0 4.7 9.1 id, a2 
Black/AA (%) 16.5 n/r 47 720 160 357 4150 20 «84 
Hisp/Lat (%) 28.4 n/r = 36.3——i16.7—S 26.2) 165268 
Wh/Caus (%) 38.1 n/r 0.2 80 490 385 468 224 591 
NtvHawaii/Pacific (%) 

7.4 n/r 51.1 n/r 0.1 0.1 0.2 n/r 0.2 
Two or More (%) 2.9 n/r a7 1.5 3.2 4.2 15 i? 3g 
Unknown (%) 0.2 n/r n/r n/r 0.1 n/r 0.5 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported. 
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Table A.5.21 Demographic Information for Grade 8 Mathematics, Overall and by State 


“Demographic =—Ssté<“<i«‘é ARC:;C;*C*éiE:CéC*C‘CS:C:CiK:sé‘<‘i‘iaR“C(‘#C‘CO‘NMDSCWNs— (GsSNNM'™—sésL:C;s 
Econ Dis (%) 50.6 99.7 503 813 503 494 423 743 53.2 
SWD (%) 18.6 161 159 217 151 2114 42259 187 231 
EL (%) 7.6 414 142 7.0 5.6 73 51 143 88 
Male (%) 51.9 500 517 507 515 529 52.7 511 53.0 
Female (%) 48.1 500 483 493 485 471 47.3 489 47.0 
AmInd/ANat (%) 13 996 08 n/r 0.3 0.3 01 107 07 
Asian (%) 4.0 n/r 2.8 1.6 4.8 3.1 5.1 08 22 
Black/AA (%) 17.8 n/r 53 767 4162 407 424188 £420 & 90 
Hisp/Lat (%) 30.1 n/r = «4.35.0 26.0 17.2 30.05.2279 
Wh/Caus (%) 37.5 n/r 0.3 51 494 347 440 195 565 
NtvHawaii/Pacific (%) 

6.4 n/r 46.1 n/r 0.1 0.1 0.2 n/r n/r 
Two or More (%) a7 n/r 3.4 14 3.0 3.9 1.2 iz 33 
Unknown (%) 0.2 n/r n/r n/r 0.1 n/r 0.6 n/r 0.3 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported; n/a = not applicable. 
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Table A.5.22 Demographic Information for Algebra |, Overall and by State 


Denieataphic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 40.7 98.9 39.3 84.0 42.1 38.5 35.1 63.6 46.6 
SWD (%) 15.6 173 12.4 17.9 4.3 147 179 1.151 141 
EL (%) 7.8 25.7 101 13.8 1.4 7.6 5.7 13.0 9.0 
Male (%) 51.5 514 527 516 493 511 513 51.5 51.5 
Female (%) 48.5 486 473 484 S07 £489 48.7 485 485 
AmInd/ANat (%) 1.4 100 0.7 n/r n/t 0.3 0.1 10.8 08 
Asian (%) 6.9 n/r 3.4 1.4 10.5 7.0 9.9 1.3 3.0 
Black/AA (%) 17.3 n/t 4.3 679 168 338 15.2 21 91 
Hisp/Lat (%) 28.7 n/r «=«.33.30 2200246169 = 27.0Ss «61.5 26.4 
Wh/Caus (%) 34.3 n/r 0.2 5.4 442 378 459 22.7 569 
NtvHawaii/Pacific (%) 

8.6 n/r 54.3 n/r n/r 0.1 0.2 n/r n/r 
Two or More (%) 2.5 n/r 3.7 2.1 3.1 4.1 13 16 33 
Unknown (%) 0.2 n/r n/r 1.0 0.5 n/r 0.4 n/r 0.3 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two or 
more races reported; n/r = not reported; n/a = not applicable. 
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Table A.5.23 Demographic Information for Geometry, Overall and by State 


Deneerahie PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 36.6 98.0 179 765 n/r 243 331 589 249 
SWD (%) 15.1 121 43 19.2 n/r 94 178 130 52 
EL (%) 5.1 18.6 1.8 8.9 n/r 25 43 102 3.0 
Male (%) 51.0 452 511 507 55.7 531 51.0 505 47.4 
Female (%) 49.0 548 489 493 443 469 490 495 52.6 
AmInd/ANat (%) 2.0 985 04 n/r n/r 0.2 01 10.7 nfr 
Asian (%) 8.0 n/r 5.7 12: 939 3232 Ge 14 39 
Black/AA (%) 13.5 n/r 21 692 nfr 196 147 18 57 
Hisp/Lat (%) 29.3 nf «176 ~=StC«a7 (sir (sisi HDSs«éCALSCi«idB; 
Wh/Caus (%) 39.5 n/r 0.2 77 67.0 509 475 234 73.7 
NtvHawaii/Pacific (%) 

5.6 n/r 69.8 n/r n/r n/r 0.2 n/r n/r 
Two or More (%) 1.8 n/r 41 2.7 cal 5.5 11 17 23 
Unknown (%) 0.3 n/r n/r 1.8 n/r n/r 0.4 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two 
or more races reported; n/r = not reported due to n<20; n/a = not applicable. 
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Table A.5.24 Demographic Information for Algebra II, Overall and by State 


Deneeraphic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 35.5 97.3 9.4 30.6 n/r 30.2 328 S545 191 
SWD (%) 12.6 10.9 3.5 11.3 n/r 97 146 101 89 
EL (%) 3.7 11.5 n/r n/r n/r 1.4 3.7 7.0 n/r 
Male (%) 50.1 520 565 471 544 484 506 495 47.1 
Female (%) 49.9 48.0 435 529 456 516 494 50.5. 52.9 
Amind/ANat (%) 2.1 99.5 n/r n/r n/r 0.3 0.1 10.2 n/r 
Asian (%) 8.6 n/r 9.9 nfr 17S isi AS 
Black/AA (%) 17.4 n/r 12 278 nfr 379 +150 21°» ofr 
Hisp/Lat (%) 27:3 n/r = 106 19.9 sont 94 260 587 nfr 
Wh/Caus (%) 40.9 n/r n/r -42.6——siA.D—iA.D——siSLTsi5D7B_T 
NtvHawaii/Pacific (%) 

1.5 n/r 73.8 n/r n/r 0.1 0.2 n/r n/r 
Two or More (%) 1.8 n/r 4.2 n/r n/r 3.8 1.1 1.7 n/r 
Unknown (%) 0.3 n/r n/r n/r n/r n/r 0.6 n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two 
or more races reported; n/r = not reported due to n<20; n/a = not applicable. 
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Table A.5.25 Demographic Information for Integrated Mathematics |, Overall and by State 


Demographic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 52.8 95.2 51.7 n/r n/r n/r n/r 63.6 n/r 
SWD (%) 12.4 n/r 12.0 n/r n/r n/r n/r 15.7 n/r 
EL (%) 20.0 n/r 20.1 n/r n/r n/r n/r 18.2 n/r 
Male (%) 53.5 n/r 53.1 n/r n/r n/r n/r 57.0 n/r 
Female (%) 46.5 n/r 46.9 n/r n/r n/r n/r 43.0 n/r 
Amind/ANat (%) 1.4 100 0.9 n/r n/r n/r n/r 3.3 n/r 
Asian (%) 3.9 n/r 4.3 n/r n/r n/r n/r n/r n/r 
Black/AA (%) 7.4 n/r 7.7 n/r n/r n/r n/r 47 n/r 
Hisp/Lat (%) 48.1 n/r 46.0 n/r n/r n/r n/r 69.2 n/r 
Wh/Caus (%) 2.6 n/r 0.3 n/r n/r n/r n/r 22.4 n/r 
NtvHawaii/Pacific (%) 

34.1 n/r 37.9 n/r n/r n/r n/r n/r n/r 
Two or More (%) 2.5 n/r 2.8 n/r n/r n/r n/r n/r n/r 
Unknown (%) n/r n/r n/r n/r n/r n/r n/r n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two 
or more races reported; n/r = not reported due to n<20; n/a = not applicable. 
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Table A.5.26 Demographic Information for Integrated Mathematics II, Overall and by State 


Danceranhie PARCC BIE CO DC lt MD NS. NM Ri 
Econ Dis (%) 46.3 99.5 33.6 41.7 n/r n/r n/r 84.1 n/r 
SWD (%) 6.6 13.6 3.0 n/r n/r n/r n/r 18.0 n/r 
EL (%) 7.7 42.4 3.1 n/r n/r n/r n/r 12.8 n/r 
Male (%) 50.9 51.5 50.4 49.6 n/r n/r n/r 50.9 n/r 
Female’) 49.1 485 496 504  n/r  nfr nfr 49.1 nf 
Amind/ANat (%) 9.3 99.0 n/r n/r n/r n/r n/r n/r n/r 
Asian (%) 49 n/r 6.2 n/r n/r n/r n/r n/r n/r 
Black/AA (%) 8.5 n/r 7.3 58.3 n/r n/r n/r n/r n/r 
Hisp/Lat (%) 28.7 n/r 25.7 n/r n/r n/r n/r 75.8 n/r 
Wh/Caus (%) 4.8 n/r n/r 22.6 n/r n/r n/r 20.4 n/r 
NtvHawaii/Pacific (%) 

40.5 n/r 55.8 n/r n/r n/r n/r n/r n/r 
Two or More (%) 3.2 n/r 4.2 n/r n/r n/r n/r n/r n/r 
Unknown (%) n/r n/r n/r n/r n/r n/r n/r n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two 
or more races reported; n/r = not reported due to n<20; n/a = not applicable. 
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Table A.5.27 Demographic Information for Integrated Mathematics III, Overall and by State 


Demographic PARCC BIE co DC IL MD NJ NM RI 
Econ Dis (%) 66.4 n/t n/r n/r n/r n/r n/r 96.5 n/r 
SWD (%) 3.8 n/r n/r n/r n/r n/r n/r 12.1 n/r 
EL (%) 6.9 n/r n/r n/r n/r n/r n/r 10.0 n/r 
Male (%) 51.8 n/r 54.2 n/r n/r n/r n/r 50.1 n/r 
Female (%) 48.2 n/r 45.8 n/r n/r n/r n/r 49.9 n/r 
Amind/ANat (%) 19.2 n/r N/r n/r n/r n/r n/r 28.0 n/r 
Asian (%) n/r n/r n/r n/r n/r n/r n/r n/r n/r 
Black/AA (%) n/r n/r n/r n/r n/r n/r n/r n/r n/r 
Hisp/Lat (%) 36.8 n/r n/r n/r n/r n/r n/r 53.1 n/r 
Wh/Caus (%) 10.6 n/r n/r n/r n/r n/r n/r 15.9 n/r 
NtvHawaii/Pacific (%) ae hi 36.3 oe i age nfr n/r n/r 
Two or More (%) n/r n/r n/r n/r n/r n/r n/r n/r n/r 
Unknown (%) n/r n/r n/r n/r n/r n/r n/r n/r n/r 


Note: PARCC = data from all participating states combined. Econ Dis = Economically Disadvantaged; SWD = Student with 
Disabilities; EL = English learner; AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific Islander; Two or More = two 
or more races reported; n/r = not reported due to n<20; n/a = not applicable. 
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Appendix 7: Summary of Differential Item Function (DIF) Results 


Table A.7.1 Differential Item Functioning for ELA/L Grade 3 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
ane eee % of Total % of Total % of Total % of Total % of Total 
o of Tota of Tota of Tota of Tota of Tota 
DIF Comparisons Mode i Item Occurrences N of ° o 7 N of ° N of ° N of 5 
Unique ; Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences. Occurrences | Occurrences | Occurrences’. Occurrences’. 
Items if in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 

Male vs Female CBT 90 105 105 1 1 102 97 2 2 
PBT 29 35 35 35 100 
White vs Amerindian CBT 90 105 105 1 1 104 99 
PBT 29 35 35 2 6 33 94 

White vs Asian CBT 90 105 105 104 99 1 1 

PBT 29 35 35 34 97 1 3 
White vs Black CBT 90 105 105 1 1 104 99 
PBT 29 35 35 1 3 34 97 
White vs Hispanic CBT 90 105 105 105 100 
PBT 29 35 35 3 9 32 91 
Whits ys Peers CBT 90 105 105 2 2 103 98 

Islander 

PBT 29 35 35 33 94 
White vs Multiracial CBT 90 105 105 105 100 
PBT 29 35 35 35 100 
NoEcnDis vs EcnDis CBT 90 105 105 105 100 
PBT 29 35 35 35 100 
ELN vs ELY CBT 90 105 105 2 2 103 98 
PBT 29 35 35 35 100 
SWDN vs SWDY CBT 90 105 105 105 100 
PBT 29 35 35 1 3 34 97 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.2 Differential Item Functioning for ELA/L Grade 4 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
: oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode E Item Occurrences N of N of N of N of N of 
Unique . Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Includedin Occurrences Occurrences’. Occurrences . Occurrences. Occurrences’. 
Items . in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 94 111 111 2 2 103 93 6 5 
PBT 31 35 35 34 97 1 3 
White vs Amerindian CBT 94 111 111 1 1 110 99 
PBT 31 35 35 1 3 1 3 33 94 
White vs Asian CBT 94 111 111 1 1 110 99 
PBT 31 35 35 1 3 34 97 
White vs Black CBT 94 111 111 111 100 
PBT 31 35 35 1 3 1 3 33 94 
White vs Hispanic CBT 94 111 111 1 2 110 99 
PBT 31 35 35 1 3 34 97 
te aed CBT 94 111 111 3 3 107 96 1 it 
Islander 
PBT 31 35 35 35 100 
White vs Multiracial CBT 94 111 111 111 100 
PBT 31 35 35 35 100 
NoEcnDis vs EcnDis CBT 94 111 111 111 100 
PBT 31 35 35 1 3 34 97 
ELN vs ELY CBT 94 111 111 111 100 
PBT 31 35 35 4 11 31 89 
SWDN vs SWDY CBT 94 111 111 111 100 
PBT 31 35 35 35 100 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disabilit 
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Table A.7.3 Differential Item Functioning for ELA/L Grade 5 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
. oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode | Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences : Occurrences’. Occurrences’. Occurrences Occurrences’. 
Items 5 in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 

Male vs Female CBT 93 109 109 1 1 3 3 98 90 7 6 

PBT 32 32 32 30 94 2 6 
White vs Amerindian CBT 93 109 109 2 2 107 98 
PBT 32 32 32 5 16 3 9 24 75 
White vs Asian CBT 93 109 109 1 1 108 99 
PBT 32 32 32 32 100 
White vs Black CBT 93 109 109 1 1 108 99 
PBT 32 32 32 32 100 
White vs Hispanic CBT 93 109 109 2 2 107 98 
PBT 32 32 32 1 3 31 97 

nee acme CBT 93 109 109 1 1 5 5 100 92 3 3 

Islander 

PBT 32 32 32 32 100 
White vs Multiracial CBT 93 109 109 109 100 
PBT 32 32 32 32 100 
NoEcnDis vs EcnDis CBT 93 109 109 109 100 
PBT 32 32 32 32 100 
ELN vs ELY CBT 93 109 109 2 2 3 3 104 95 
PBT 32 32 32 dl. 3 1 3 30 94 
SWDN vs SWDY CBT 93 109 109 109 100 
PBT 32 32 32 32 100 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.4 Differential Item Functioning for ELA/L Grade 6 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
. oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode | Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences : Occurrences’ . Occurrences’. Occurrences Occurrences’. 
Items 5 in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 113 129 129 1 1 8 6 aA1 86 9 7 
PBT 38 42 42 1 2 5 12 34 81 2 5 
White vs Amerindian CBT 113 129 129 7 5 117 91 5 4 
PBT 38 42 42 3 7 4 10 35 83 
White vs Asian CBT 113 129 129 1 1 128 99 
PBT 38 42 42 1 2 36 86 5 12 
White vs Black CBT 113 129 129 129 100 
PBT 38 42 42 1 2 41 98 
White vs Hispanic CBT 113 129 129 1 1 128 99 
PBT 38 42 42 1 2 1 2 40 95 
nee acme cBT = 113 129 86 5 6 76 88 5 6 
Islander 
PBT 38 42 42 42 100 
White vs Multiracial CBT 113 129 129 129 100 
PBT 38 42 42 42 100 
NoEcnDis vs EcnDis CBT 113 129 129 129 100 
PBT 38 42 42 42 100 
ELN vs ELY CBT 113 129 129 2 2 127 98 
PBT 38 42 42 4 10 38 90 
SWDN vs SWDY CBT 113 129 129 129 100 
PBT 38 42 42 2 5 40 95 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.5 Differential Item Functioning for ELA/L Grade 7 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
. oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode | Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences : Occurrences’ . Occurrences’. Occurrences Occurrences’. 
Items a in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 105 129 129 1 1 11. 9 105 81 12 9 
PBT 36 44 44 3 7 40 91 1 2 
White vs Amerindian CBT 105 129 129 3 2 126 98 
PBT 36 44 44 2 5 3 7 37 84 2 5 
White vs Asian CBT 105 129 129 2 2 127 98 
PBT 36 44 44 1 2 40 91 3 7 
White vs Black CBT 105 129 129 1 1 128 99 
PBT 36 44 44 2 5 42 95 
White vs Hispanic CBT 105 129 129 2 2 127 98 
PBT 36 44 44 1 2 1 2 42 95 
nee acme CBT 105 129 43 1 2 38 88 4 9 
Islander 
PBT 36 44 44 2 5 42 95 
White vs Multiracial CBT 105 129 129 129 100 
PBT 36 44 44 44 100 
NoEcnDis vs EcnDis CBT 105 129 129 129 100 
PBT 36 44 44 44 100 
ELN vs ELY CBT 105 129 129 3 2 3 2 118 91 5 4 
PBT 36 44 44 d. 2 1 2 41 93 1 2 
SWDN vs SWDY CBT 105 129 129 129 100 
PBT 36 44 44 44 100 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.6 Differential Item Functioning for ELA/L Grade 8 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
. oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode | Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences : Occurrences’ . Occurrences . Occurrences Occurrences’. 
Items a in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 112 129 129 11 9 115 89 3 2 
PBT 36 50 50 2 4 8 16 40 80 
White vs Amerindian CBT 112 129 129 1 1 9 7 117 91 2 2 
PBT 36 50 50 6 12 3 6 41 82 
White vs Asian CBT 112 129 129 129 100 
PBT 36 50 50 45 90 5 10 
White vs Black CBT 112 129 129 1 1 128 99 
PBT 36 50 50 1 2 49 98 
White vs Hispanic CBT 112 129 129 1 1 128 99 
PBT 36 50 50 2 4 48 96 
nee acme cBT 112 129 43 1 2 41 95 1 2 
Islander 
PBT 36 50 50 50 100 
White vs Multiracial CBT 112 129 129 129 100 
PBT 36 50 50 50 100 
NoEcnDis vs EcnDis CBT 112 129 129 129 100 
PBT 36 50 50 50 100 
ELN vs ELY CBT 112 129 129 3 2 4 3 122 95 
PBT 36 50 50 d. 2 2 4 46 92 1 2 
SWDN vs SWDY CBT 112 129 129 129 100 
PBT 36 50 50 50 100 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.7 Differential Item Functioning for ELA/L Grade 9 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
. oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode | Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences : Occurrences’. Occurrences’. Occurrences Occurrences’. 
Items a in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 102 129 129 4 3 9 vi 111 86 5 4 
PBT 36 53 53 4 8 48 91 1 2 
White vs Amerindian CBT 102 129 129 1 1 4 3 124 96 
PBT 36 53 53 3 6 2 4 48 91 
White vs Asian CBT 102 129 129 1 1 128 99 
PBT 36 53 53 53 100 
White vs Black CBT 102 129 129 4 3 125 97 
PBT 36 53 53 1. 2 1 2 51 96 
White vs Hispanic CBT 102 129 129 2 2 127 98 
PBT 36 53 53 2 4 51, 96 
nee acme cBT 102 129 129 4 3 125 97 
Islander 

PBT 36 53 53 53 100 
White vs Multiracial CBT 102 129 129 129 100 
PBT 36 53 53 1 2 52 98 
NoEcnDis vs EcnDis CBT 102 129 129 129 100 
PBT 36 53 53 53 100 
ELN vs ELY CBT 102 129 129 1 1 128 99 
PBT 36 53 53 2 4 4 8 47 89 
SWDN vs SWDY CBT 102 129 129 129 100 
PBT 36 53 53 1 2 52 98 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.8 Differential Item Functioning for ELA/L Grade 10 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
. oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode | Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences : Occurrences’ . Occurrences’. Occurrences Occurrences’. 
Items a in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 112 129 129 10 8 115 89 4 3 
PBT 36 42 42 3 7 39 93 
White vs Amerindian CBT 112 129 129 3 2 3 2 116 90 7d. 5 
PBT 36 42 42 1 2 3 7 38 90 
White vs Asian CBT 112 129 129 1 1 128 99 
PBT 36 42 42 2 5 40 95 
White vs Black CBT 112 129 129 3 2 126 98 
PBT 36 42 42 42 100 
White vs Hispanic CBT 112 129 129 1 1 1 1 127 98 
PBT 36 42 42 1 2 41 98 
White vs Pacific CBT 112 129 0 
Islander 

PBT 36 42 42 42 100 
White vs Multiracial CBT 112 129 129 1 1 128 99 
PBT 36 42 42 2 5 40 95 
NoEcnDis vs EcnDis CBT 112 129 129 129 100 
PBT 36 42 42 42 100 
ELN vs ELY CBT 112 129 129 2 2 2 2 125 97 
PBT 36 42 42 1 2 2 5 39 93 
SWDN vs SWDY CBT 112 129 129 129 100 
PBT 36 42 42 42 100 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.9 Differential Item Functioning for ELA/L Grade 11 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
| oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode | Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences : Occurrences. Occurrences’. Occurrences Occurrences’. 
Items a in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 114 129 129 5 4 121 94 3 2 
PBT 34 50 50 4 8 46 92 
White vs Amerindian CBT 114 129 129 9 7 15 12 90 70 6 5 9 7 
PBT 34 50 50 1 2 9 18 40 80 
White vs Asian CBT 114 129 129 129 100 
PBT 34 50 50 1 2 47 94 2 4 
White vs Black CBT 114 129 129 3 2 125 97 1 1 
PBT 34 50 50 50 100 
White vs Hispanic CBT 114 129 129 2 2 7 5 115 89 5 4 
PBT 34 50 50 1 2 49 98 
White vs Pacific CBT 414 129 0 
Islander 
PBT 34 50 50 2 4 48 96 
White vs Multiracial CBT 114 129 129 1 1 128 99 
PBT 34 50 50 50 100 
NoEcnDis vs EcnDis CBT 114 129 129 4 3 125 97 
PBT 34 50 50 50 100 
ELN vs ELY CBT 114 129 129 6 5 4 3 111 86 8 6 
PBT 34 50 50 1 2 2 4 47 94 
SWDN vs SWDY CBT 114 129 129 129 100 
PBT 34 50 50 2 4 47 94 1 2 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.10 Differential Item Functioning for Mathematics Grade 3 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
. oO % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode : Item Occurrences N of N of N of N of N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Includedin Occurrences Occurrences . Occurrences. Occurrences : Occurrences’. 
Items in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 113 124 124 3 2 120 97 1 1 
PBT 43 63 63 1 2 62 98 
White vs AmerIndian CBT 113 124 124 3 2 1 1 118 95 2 2 
PBT 43 63 63 1 2 3 5 57 90 2 3 
White vs Asian CBT 113 124 124 1 a 107 86 13 10 3 2 
PBT 43 63 63 60 95 2 3 1 2 
White vs Black CBT 113 124 124 5 4 5 4 105 85 9 7 
PBT 43 63 63 3 5 58 92 2 3 
White vs Hispanic CBT 113 124 124 4 3 119 96 1 1 
PBT 43 63 63 3 5 58 92 2 3 
Wneaye 2 Seni cBT 113 124 124 123 99 1 1 
Islander 
PBT 43 63 63 59 94 4 6 
White vs Multiracial CBT 113 124 124 4 1 121 98 2 2 
PBT 43 63 63 62 98 1 2 
NoEcnDis vs EcnDis CBT 113 124 124 123 99 1 i 
PBT 43 63 63 61 97 1 2 1 2 
ELN vs ELY CBT 113 124 124 2 2 122 98 
PBT 43 63 63 1 2 62 98 
SWDN vs SWDY CBT 113 124 124 2 2 3 2 117 94 2 2 
PBT 43 63 63 1 2 61 97 1 2 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
Z % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of oO 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences Occurrences Occurrences Occurrences Occurrences 7 
Items é n DIF n DIF n DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 99 104 104 6 6 96 92 2 2 
PBT 40 61 61 3 5 58 95 
White vs AmerIndian CBT 99 104 104 4 4 7 7 93 89 
PBT 40 61 61 2 3 4 7 53 87 2 3 
White vs Asian CBT 99 104 104 98 94 5 5 1 1 
PBT 40 61 61 1 2 58 95 1 2 a 2 
White vs Black CBT 99 104 104 1 1 8 8 95 91 
PBT 40 61 61 1 2 4 7 54 89 2 3 
White vs Hispanic CBT 99 104 104 3 3 101 97 
PBT 40 61 61 2 3 59 97 
ania ver een CBT 99 104 104 104 100 
Islander 
PBT 40 61 61 60 98 a 2 
White vs Multiracial CBT 99 104 104 1 1 102 98 1 1 
PBT 40 61 61 60 98 1 2 
NoEcnDis vs EcnDis CBT 99 104 104 1 i 103 99 
PBT 40 61 61 4 7 56 92 1 2 
ELN vs ELY CBT 99 104 104 2 2 7 7 95 91 
PBT 40 61 61 4 7 57 93 
SWDN vs SWDY CBT 99 104 104 2 2 102 98 
PBT 40 61 61 3 5 57 93 1 2 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 


Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 


with disability. 


March 27, 2018 


Page 289 


PARCC 2017 Technical Report 


Table A.7.12 Differential Item Functioning for Mathematics Grade 5 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
Z % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of o oO oO oO 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Includedin Occurrences’. Occurrences . Occurrences. Occurrences : Occurrences’. 
Items : in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 103 113 113 3 3 109 96 1 1 
PBT 40 56 56 2 4 54 96 
White vs AmerIndian CBT 103 113 113 3 3 109 96 1 1 
PBT 40 56 56 56 100 
White vs Asian CBT 103 113 113 112 99 1 1 
PBT 40 56 56 56 100 
White vs Black CBT 103 113 113 1 1 3 3 109 96 
PBT 40 56 56 1 2 55 98 
White vs Hispanic CBT 103 113 113 113 100 
PBT 40 56 56 56 100 
ania ver een CBT 103 113 113 113 100 
Islander 
PBT 40 56 56 56 100 
White vs Multiracial CBT 103 113 113 3 3 110 97 
PBT 40 56 56 56 100 
NoEcnDis vs EcnDis CBT 103 113 113 1 1 112 99 
PBT 40 56 56 1 2 54 96 1 2 
ELN vs ELY CBT 103 113 113 2 2 3 3 108 96 
PBT 40 56 56 1 2 1 2 54 96 
SWDN vs SWDY CBT 103 113 113 3 3 107 95 1 a 2 2 
PBT 40 56 56 2 4 52 93 2 4 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Table A.7.13 Differential Item Functioning for Mathematics Grade 6 


Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
i {o} % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode : Item Occurrences N of o oO oO oO 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Includedin Occurrences Occurrences . Occurrences. Occurrences ‘ Occurrences’. 
Items f in DIF in DIF in DIF in DIF in DIF 
DIF Analysis 
Male vs Female CBT 95 103 103 3 3 98 95 2 2 
PBT 38 51 51 51 100 
White vs AmerIndian CBT 95 103 103 2 2 100 97 1 1 
PBT 38 51 51 2 4 49 96 
White vs Asian CBT 95 103 103 94 91 8 8 1 1 
PBT 38 51 51 50 98 1 2 
White vs Black CBT 95 103 103 1 1 3 3 99 96 
PBT 38 51 51 50 98 1 2 
White vs Hispanic CBT 95 103 103 2 2 101 98 
PBT 38 51 51 1 2 50 98 
ania ver een CBT 95 103 103 1 1 102 99 
Islander 
PBT 38 51 51 1 2 49 96 1 2 
White vs Multiracial CBT 95 103 103 1 1 1 1 99 96 2 2 
PBT 38 51 51 51 100 
NoEcnDis vs EcnDis CBT 95 103 103 103 100 
PBT 38 51 51 51 100 
ELN vs ELY CBT 95 103 103 1 1 6 6 96 93 
PBT 38 51 51 1 2 50 98 
SWDN vs SWDY CBT 95 103 103 2 2 7 7 92 89 2 2 
PBT 38 51 51 1 2 49 96 1 2 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 
Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 
with disability. 
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Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
Z % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of oO 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences Occurrences Occurrences Occurrences Occurrences 7 
Items é n DIF n DIF n DIF n DIF in DIF 
DIF Analysis 
Male vs Female CBT 97 103 103 2 2 5 5 96 93 
PBT 38 49 49 49 100 
White vs AmerIndian CBT 97 103 103 3 3 2 2 98 95 
PBT 38 49 49 2 4 47 96 
White vs Asian CBT 97 103 103 3 3 95 92 4 4 1 1 
PBT 38 49 49 4 8 39 80 5 10 a 2 
White vs Black CBT 97 103 103 1 1 8 8 93 90 1 1 
PBT 38 49 49 48 98 1 2 
White vs Hispanic CBT 97 103 103 2 2 101 98 
PBT 38 49 49 1 2 48 98 
ania ver een CBT 97 103 103 103 100 
Islander 
PBT 38 49 49 49 100 
White vs Multiracial CBT 97 103 103 1 1 101 98 1 1 
PBT 38 49 49 49 100 
NoEcnDis vs EcnDis CBT 97 103 103 103 100 
PBT 38 49 49 49 100 
ELN vs ELY CBT 97 103 103 1 1 11 11 89 86 2 2 
PBT 38 49 49 4 8 44 90 1 2 
SWDN vs SWDY CBT 97 103 103 2 2 99 96 2 2 
PBT 38 49 49 49 100 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 


Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 


with disability. 
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Table A.7.15 Differential Item Functioning for Mathematics Grade 8 
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Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
Z % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences Occurrences Occurrences Occurrences Occurrences 
Items F n DIF n DIF n DIF n DIF n DIF 
DIF Analysis 
Male vs Female CBT 91 100 100 2 2 98 98 
PBT 36 53 53 4 8 49 92 
White vs AmerIndian CBT 91 100 100 3 3 3 3 94 94 
PBT 36 53 53 1 2 52 98 
White vs Asian CBT 91 100 100 1 1 1 1 91 91 6 6 1 1 

PBT 36 53 53 48 91 5 9 
White vs Black CBT 91 100 100 1 1 3 3 96 96 

PBT 36 53 53 1 2 51 96 1 2 
White vs Hispanic CBT 91 100 100 1 1 1 1 98 98 
PBT 36 53 53 53 100 
ania ver een CBT 91 100 100 1 1 99 99 

Islander 

PBT 36 53 53 1 2 52 98 
White vs Multiracial CBT 91 100 100 100 100 

PBT 36 53 53 52 98 1 2 
NoEcnDis vs EcnDis CBT 91 100 100 1 1 1 1 98 98 
PBT 36 53 53 1 2 52 98 

ELN vs ELY CBT 91 100 100 1 1 3 3 95 95 1 1 
PBT 36 53 53 1 2 52 98 
SWDN vs SWDY CBT 91 100 100 1 1 3 3 96 96 
PBT 36 53 53 1 2 52 98 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 


Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 


with disability. 
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Table A.7.16 Differential Item Functioning for Mathematics Algebra | 
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Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
Z % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences Occurrences Occurrences Occurrences Occurrences 
Items é n DIF n DIF n DIF n DIF n DIF 
DIF Analysis 
Male vs Female CBT 104 108 108 3 3 103 95 2 2 
PBT 42 57 57 2 4 2 4 53 93 
White vs AmerIndian CBT 104 108 108 2 2 2 2 104 96 
PBT 42 57 57 57 100 
White vs Asian CBT 104 108 108 1 1 2 2 95 88 8 7 2 2 
PBT 42 57 57 1 2 50 88 5 9 1. 2 
White vs Black CBT 104 108 108 4 4 104 96 
PBT 42 57 57 1 2 56 98 
White vs Hispanic CBT 104 108 108 108 100 
PBT 42 57 57 57 100 
ania ver een CBT 104 108 108 106 98 2 2 
Islander 
PBT 42 57 57 1 2 55 96 1 2 
White vs Multiracial CBT 104 108 108 4 4 103 95 1 1 
PBT 42 57 57 fi 2 56 98 
NoEcnDis vs EcnDis CBT 104 108 108 2 2 106 98 
PBT 42 57 57 57 100 
ELN vs ELY CBT 104 108 108 2 2 105 97 1 1 
PBT 42 57 57 57 100 
SWDN vs SWDY CBT 104 108 108 3 3 105 97 
PBT 42 57 57 57 100 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 


Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 


with disability. 
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Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
i % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences Occurrences Occurrences Occurrences Occurrences 
Items : n DIF n DIF n DIF n DIF n DIF 
DIF Analysis 
Male vs Female CBT 110 121 121 2 2 4 3 115 95 
PBT 43 65 65 1 2 2 3 62 95 
White vs AmerIndian CBT 110 121 121 1 1 119 98 1 1 
PBT 43 65 65 2 3 1 2 62 95 
White vs Asian CBT 110 121 121 3 2 109 90 9 7 
PBT 43 65 65 1 2 58 89 4 6 2 3 
White vs Black CBT 110 121 121 6 5 112 93 2 2 1 1 
PBT 43 65 65 2 3 62 95 1 2 
White vs Hispanic CBT 110 121 121 4 3 117 97 
PBT 43 65 65 65 100 
ania ver een cBT 110 121 121 1 1 120 99 
Islander 
PBT 43 65 65 64 98 1 2 
White vs Multiracial CBT 110 121 121 2 2 119 98 
PBT 43 65 65 2 3 63 97 
NoEcnDis vs EcnDis CBT 110 121 121 4 3 117 97 
PBT 43 65 65 1 2 64 98 
ELN vs ELY CBT 110 121 121 5 4 12 10 99 82 3 2 2 2 
PBT 43 65 65 4 6 60 92 1 2 
SWDN vs SWDY CBT 110 121 121 2 2 9 7 109 90 1 1 
PBT 43 65 65 1 2 4 6 59 91 1 2 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 


Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 


with disability. 
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Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
Z % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences Occurrences Occurrences Occurrences Occurrences 
Items . n DIF n DIF n DIF n DIF n DIF 
DIF Analysis 
Male vs Female CBT 102 111 111 1 1 4 4 106 95 
PBT 41 59 59 59 100 
White vs AmerIndian CBT 102 111 111 111 100 
PBT 41 59 59 59 100 
White vs Asian CBT 102 111 111 1 1 85 77 20 18 5 5 
PBT 41 59 59 1 2 1 2 54 92 3 5 
White vs Black CBT 102 111 111 2 2 2 2 107 96 
PBT 41 59 59 2 3 56 95 1 2 
White vs Hispanic CBT 102 111 111 1 1 110 99 
PBT 41 59 59 3 5 56 95 
ania ver een CBT 102 111 111 111 100 
Islander 
PBT 41 59 59 59 100 
White vs Multiracial CBT 102 111 111 111 100 
PBT 41 59 59 2 3 57 97 
NoEcnDis vs EcnDis CBT 102 111 111 1 1 110 99 
PBT 41 59 59 1 2 58 98 
ELN vs ELY CBT 102 111 111 1 1 4 4 105 95 1 1 
PBT 41 59 59 1 2 58 98 
SWDN vs SWDY CBT 102 111 111 6 5 103 93 2 2 
PBT 41 59 59 3 5 55 93 1 2 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 


Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 


with disability. 
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Total N of C- DIF B- DIF A DIF B+ DIF C+ DIF 
Total N 
Total N of Item 
i % of Total % of Total % of Total % of Total % of Total 
DIF Comparisons Mode ; Item Occurrences N of 
Unique : Occurrences Occurrences Occurrences Occurrences Occurrences 
Occurrences Included in Occurrences Occurrences Occurrences Occurrences Occurrences 
Items F n DIF n DIF n DIF n DIF n DIF 
DIF Analysis 
Male vs Female CBT 42 51 51 2 4 4 8 44 86 1 2 
PBT 42 64 64 2 3 61 95 1 2 
White vs AmerIndian CBT 42 51 51 51 100 
PBT 42 64 64 1 2 63 98 
White vs Asian CBT 42 51 51 1 2 1 2 45 88 3 6 1 2 
PBT 42 64 64 1 2 62 97 1 2 
White vs Black CBT 42 51 51 3 6 48 94 
PBT 42 64 64 1 2 63 98 
White vs Hispanic CBT 42 51 51 2 4 49 96 
PBT 42 64 64 1 2 2 3 61 95 
ania ver een CBT 42 51 51 51 100 
Islander 
PBT 42 64 64 64 100 
White vs Multiracial CBT 42 51 51 1 2 50 98 
PBT 42 64 64 1 2 62 97 1 2 
NoEcnDis vs EcnDis CBT 42 51 51 1 2 50 98 
PBT 42 64 64 63 98 1 2 
ELN vs ELY CBT 42 51 51 3 6 2 4 46 90 
PBT 42 64 64 4 6 2 3 58 91 
SWDN vs SWDY CBT 42 51 51 2 4 49 96 
PBT 42 64 64 1 2 4 6 59 92 


Note: Amerindian = American Indian/Alaska Native, Black = Black/African American, Hispanic = Hispanic/Latino, Pacific Islander = Native Hawaiian or Pacific Islander, Multiracial = Multiple Race 


Selected, NoEcnDis = not economically disadvantaged, EcnDis = economically disadvantaged, ELN = not an English learner, ELY = English learner, SWDN = not student with disability, SWDY = student 


with disability. 
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Appendix 8: Reliability of Classification by Content and Grade Level 


Table A.8.1 Reliability of Classification: Grade 3 ELA/L 


Full Summative Category 
eigen Level 1 Level 2 Level 3 Level 4 Level 5 Total 
650 - 699 0.15 0.02 0.00 0.00 0.00 0.18 
Decision 700 - 724 0.03 0.11 0.04 0.00 0.00 0.19 
Accuracy 725 - 749 0.00 0.04 0.15 0.04 0.00 0.23 
750 - 809 0.00 0.00 0.04 0.31 0.02 0.37 
ear 810 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 — 699 0.15 0.04 0.00 0.00 0.00 0.19 
Decision 700 - 724 0.04 0.09 0.05 0.01 0.00 0.19 
Consistency 725 - 749 0.00 0.04 0.11 0.05 0.00 0.22 
750 - 809 0.00 0.01 0.06 0.28 0.02 0.37 
810 - 850 0.00 0.00 0.00 0.02 0.02 0.04 
650 — 699 0.17 0.03 0.00 0.00 0.00 0.19 
Decision 700 - 724 0.04 0.11 0.04 0.00 0.00 0.20 
Accuracy 725 - 749 0.00 0.04 0.12 0.05 0.00 0.22 
750 - 809 0.00 0.00 0.04 0.29 0.02 0.36 
810 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
aa 650 — 699 0.16 0.04 0.01 0.00 0.00 0.21 
Decision 700 - 724 0.04 0.09 0.05 0.01 0.00 0.19 
Consistency 725 - 749 0.01 0.05 0.09 0.06 0.00 0.21 
750 - 809 0.00 0.01 0.06 0.26 0.02 0.35 
810 - 850 0.00 0.00 0.00 0.02 0.02 0.04 
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Table A.8.2 Reliability of Classification: Grade 4 ELA/L 


Full Summative Category 
Seabee enie Level 1 Level 2 Level 3 Level 4 Level 5 rata 
650 — 699 0.10 0.02 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.03 0.12 0.04 0.00 0.00 0.19 
Accuracy 725 - 749 0.00 0.04 0.17 0.05 0.00 0.26 
750 - 789 0.00 0.00 0.05 0.27 0.03 0.35 
ear 790 - 850 0.00 0.00 0.00 0.02 0.07 0.08 
650 — 699 0.09 0.03 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.03 0.10 0.06 0.00 0.00 0.19 
Consistency 725 - 749 0.00 0.05 0.14 0.06 0.00 0.25 
750 - 789 0.00 0.00 0.06 0.24 0.03 0.34 
790 - 850 0.00 0.00 0.00 0.04 0.06 0.10 
650 — 699 0.11 0.02 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.03 0.13 0.05 0.00 0.00 0.21 
Accuracy 725 - 749 0.00 0.05 0.19 0.05 0.00 0.29 
750 - 789 0.00 0.00 0.06 0.24 0.03 0.33 
790 - 850 0.00 0.00 0.00 0.01 0.03 0.04 
as 650 - 699 0.10 0.04 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.10 0.07 0.01 0.00 0.21 
Consistency 725 - 749 0.00 0.05 0.15 0.06 0.00 0.27 
750 - 789 0.00 0.01 0.08 0.21 0.03 0.32 
790 - 850 0.00 0.00 0.00 0.03 0.03 0.05 
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Table A.8.3 Reliability of Classification: Grade 5 ELA/L 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.09 0.02 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.03 0.13 0.04 0.00 0.00 0.19 
Accuracy 725 - 749 0.00 0.04 0.17 0.05 0.00 0.26 
750 - 798 0.00 0.00 0.05 0.33 0.02 0.40 
ear 799 - 850 0.00 0.00 0.00 0.01 0.03 0.04 
650 - 699 0.09 0.03 0.00 0.00 0.00 0.12 
Decision 700 - 724 0.03 0.10 0.05 0.00 0.00 0.19 
Consistency 725 - 749 0.00 0.05 0.14 0.06 0.00 0.25 
750 - 798 0.00 0.00 0.06 0.30 0.02 0.39 
799 - 850 0.00 0.00 0.00 0.02 0.03 0.05 
650 - 699 0.09 0.02 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.03 0.14 0.04 0.00 0.00 0.22 
Accuracy 725 - 749 0.00 0.05 0.19 0.05 0.00 0.29 
750 - 798 0.00 0.00 0.05 0.29 0.02 0.37 
799 - 850 0.00 0.00 0.00 0.00 0.01 0.01 
as 650 - 699 0.09 0.03 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.04 0.11 0.06 0.01 0.00 0.21 
Consistency 725 - 749 0.00 0.06 0.15 0.07 0.00 0.28 
750 - 798 0.00 0.01 0.07 0.27 0.02 0.36 
799 - 850 0.00 0.00 0.00 0.01 0.01 0.03 
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Table A.8.4 Reliability of Classification: Grade 6 ELA/L 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.08 0.01 0.00 0.00 0.00 0.09 
Decision 700 - 724 0.02 0.16 0.03 0.00 0.00 0.22 
Accuracy 725 - 749 0.00 0.04 0.21 0.04 0.00 0.29 
750 - 789 0.00 0.00 0.04 0.28 0.02 0.34 
ear 790 - 850 0.00 0.00 0.00 0.01 0.05 0.06 
650 - 699 0.08 0.03 0.00 0.00 0.00 0.10 
Decision 700 - 724 0.03 0.13 0.05 0.00 0.00 0.21 
Consistency 725 - 749 0.00 0.05 0.18 0.06 0.00 0.28 
750 - 789 0.00 0.00 0.06 0.26 0.02 0.33 
790 - 850 0.00 0.00 0.00 0.02 0.05 0.07 
650 — 699 0.09 0.01 0.00 0.00 0.00 0.10 
Decision 700 - 724 0.02 0.14 0.03 0.00 0.00 0.20 
Accuracy 725 - 749 0.00 0.04 0.20 0.04 0.00 0.29 
750 - 789 0.00 0.00 0.04 0.30 0.02 0.37 
790 - 850 0.00 0.00 0.00 0.01 0.03 0.05 
as 650 - 699 0.08 0.03 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.03 0.12 0.05 0.00 0.00 0.20 
Consistency 725 - 749 0.00 0.05 0.17 0.06 0.00 0.28 
750 - 789 0.00 0.00 0.06 0.27 0.03 0.36 
790 - 850 0.00 0.00 0.00 0.03 0.03 0.06 
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Table A.8.5 Reliability of Classification: Grade 7 ELA/L 


Full Summative Category 
Sabena Level 1 Level 2 Level 3 Level 4 Level 5 rete 
650 — 699 0.11 0.02 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.03 0.11 0.04 0.00 0.00 0.18 
Accuracy 725 - 749 0.00 0.03 0.16 0.04 0.00 0.24 
750 - 784 0.00 0.00 0.04 0.24 0.03 0.31 
785 - 850 0.00 0.00 0.00 0.02 0.11 0.14 
ia 650 - 699 0.11 0.03 0.00 0.00 0.00 0.14 
Decision 700 - 724 0.03 0.09 0.05 0.00 0.00 0.18 
Consistency 725 - 749 0.00 0.04 0.13 0.06 0.00 0.23 
750 - 784 0.00 0.00 0.06 0.21 0.03 0.30 
785 - 850 0.00 0.00 0.00 0.04 0.11 0.15 
650 — 699 0.12 0.02 0.00 0.00 0.00 0.14 
Decision 700 - 724 0.03 0.12 0.04 0.00 0.00 0.19 
Accuracy 725 - 749 0.00 0.04 0.16 0.05 0.00 0.25 
750 - 784 0.00 0.00 0.05 0.23 0.03 0.31 
785 - 850 0.00 0.00 0.00 0.03 0.09 0.11 
as 650 - 699 0.12 0.03 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.09 0.06 0.01 0.00 0.19 
Consistency 725 - 749 0.00 005 0.13 0.06 0.00 0.24 
750 - 784 0.00 0.00 0.06 0.19 0.04 0.30 
785 - 850 0.00 0.00 0.00 0.04 0.08 0.13 
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Table A.8.6 Reliability of Classification: Grade 8 ELA/L 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.12 0.02 0.00 0.00 0.00 0.14 
Decision 700 - 724 0.03 0.12 0.04 0.00 0.00 0.19 
Accuracy 725 - 749 0.00 0.04 0.16 0.04 0.00 0.24 
750 - 793 0.00 0.00 0.04 0.28 0.02 0.34 
ear 794 - 850 0.00 0.00 0.00 0.02 0.07 0.09 
650 - 699 0.12 0.03 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.10 0.05 0.00 0.00 0.18 
Consistency 725 - 749 0.00 0.04 0.13 0.06 0.00 0.23 
750 - 793 0.00 0.00 0.06 0.25 0.03 0.33 
794 - 850 0.00 0.00 0.00 0.03 0.07 0.10 
650 - 699 0.12 0.02 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.14 0.05 0.00 0.00 0.22 
Accuracy 725 - 749 0.00 0.05 0.17 0.05 0.00 0.26 
750 - 793 0.00 0.00 0.05 0.24 0.02 0.31 
794 - 850 0.00 0.00 0.00 0.01 0.05 0.07 
as 650 - 699 0.12 0.04 0.00 0.00 0.00 0.16 
Decision 700 - 724 0.03 0.11 0.06 0.01 0.00 0.21 
Consistency 725 - 749 0.00 0.05 0.14 0.06 0.00 0.25 
750 - 793 0.00 0.01 0.07 0.21 0.03 0.31 
794 - 850 0.00 0.00 0.00 0.03 0.05 0.08 
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Table A.8.7 Reliability of Classification: Grade 9 ELA/L 


Full Summative Category 
Seabee enie Level 1 Level 2 Level 3 Level 4 Level 5 satel 
650 — 699 0.13 0.02 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.12 0.03 0.00 0.00 0.18 
Accuracy 725 - 749 0.00 0.03 0.17 0.04 0.00 0.24 
750 - 790 0.00 0.00 0.04 0.27 0.02 0.34 
ear 791 - 850 0.00 0.00 0.00 0.02 0.07 0.09 
650 - 699 0.13 0.03 0.00 0.00 0.00 0.16 
Decision 700 - 724 0.03 0.10 0.05 0.00 0.00 0.18 
Consistency 725 - 749 0.00 0.04 0.14 0.05 0.00 0.23 
750 - 790 0.00 0.00 0.05 0.25 0.03 0.33 
791 - 850 0.00 0.00 0.00 0.03 0.07 0.10 
650 — 699 0.21 0.03 0.00 0.00 0.00 0.24 
Decision 700 - 724 0.04 0.15 0.04 0.00 0.00 0.23 
Accuracy 725 - 749 0.00 0.04 0.16 0.04 0.00 0.24 
750 - 790 0.00 0.00 0.04 0.19 0.02 0.24 
791 - 850 0.00 0.00 0.00 0.01 0.04 0.05 
as 650 - 699 0.21 0.04 0.00 0.00 0.00 0.25 
Decision 700 - 724 0.04 0.12 0.06 0.00 0.00 0.22 
Consistency 725 - 749 0.00 0.05 0.13 0.05 0.00 0.23 
750 - 790 0.00 0.00 0.05 0.17 0.02 0.24 
791 - 850 0.00 0.00 0.00 0.02 0.04 0.06 
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Table A.8.8 Reliability of Classification: Grade 10 ELA/L 


Full Summative Category 
mabe ceaie Level 1 Level 2 Level 3 Level 4 Level 5 ‘ratel 
650 — 699 0.18 0.02 0.00 0.00 0.00 0.20 
Decision 700 - 724 0.03 0.09 0.04 0.00 0.00 0.15 
Accuracy 725 - 749 0.00 0.03 0.11 0.04 0.00 0.19 
750 - 793 0.00 0.00 0.04 0.24 0.03 0.32 
ear 794 - 850 0.00 0.00 0.00 0.03 0.11 0.14 
650 - 699 0.17 0.03 0.01 0.00 0.00 0.21 
Decision 700 - 724 0.03 0.07 0.05 0.01 0.00 0.15 
Consistency 725 - 749 0.00 0.04 0.09 0.05 0.00 0.18 
750 - 793 0.00 0.01 0.05 0.21 0.04 0.31 
794 - 850 0.00 0.00 0.00 0.05 0.10 0.15 
650 — 699 0.21 0.02 0.00 0.00 0.00 0.23 
Decision 700 - 724 0.03 0.09 0.04 0.00 0.00 0.16 
Accuracy 725 - 749 0.00 0.03 0.11 0.04 0.00 0.18 
750 - 793 0.00 0.00 0.04 0.22 0.03 0.29 
794 - 850 0.00 0.00 0.00 0.02 0.12 0.14 
as 650 - 699 0.20 0.04 0.00 0.00 0.00 0.24 
Decision 700 - 724 0.04 0.07 0.04 0;01 0.00 0.16 
Consistency 725 - 749 0.00 0.04 0.08 0.05 0.00 0.17 
750 — 793 0.00 0.01 0.05 0.19 0.03 0.28 
794 - 850 0.00 0.00 0.00 0.04 0.11 0.15 
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Table A.8.9 Reliability of Classification: Grade 11 ELA/L 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.18 0.02 0.00 0.00 0.00 0.20 
Decision 700 — 724 0.03 0.13 0.04 0.00 0.00 0.20 
Accuracy 725 - 749 0.00 0.04 0.15 0.04 0.00 0.23 
750 - 791 0.00 0.00 0.04 0.24 0.02 0.30 
ear 792 - 850 0.00 0.00 0.00 0.02 0.06 0.08 
650 - 699 0.17 004 0.00 0.00 0.00 0.21 
Decision 700 - 724 0.04 0.10 0.05 0.00 0.00 0.20 
Consistency 725 - 749 0.00 0.04 0.12 0.05 0.00 0.22 
750 - 791 0.00 0.00 0.05 0.21 0.02 0.29 
792 - 850 0.00 0.00 0.00 0.03 0.06 0.08 
650 — 699 0.16 0.03 0.00 0.00 0.00 0.19 
Decision 700 - 724 0.05 0.12 0.05 0.00 0.00 0.23 
Accuracy 725 - 749 0.00 0.05 0.15 0.05 0.00 0.26 
750 - 791 0.00 0.00 0.05 0.21 0.03 0.30 
792 - 850 0.00 0.00 0.00 0.01 0.02 0.05 
as 650 — 699 0.15 0.04 0.01 0.00 0.00 0.21 
Decision 700 - 724 0.05 0.10 0.06 0.01 0.00 0.22 
Consistency 725 - 749 0.01 0.05 0.12 0.06 0.00 0.24 
750 - 791 0.00 0.01 0.07 0.18 0.03 0.29 
792 - 850 0.00 0.00 0.00 0.03 0.02 0.05 
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Table A.8.10 Reliability of Classification: Grade 3 Mathematics 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.10 0.02 0.00 0.00 0.00 0.12 
Decision 700 - 724 0.02 0.13 0.03 0.00 0.00 0.19 
Accuracy 725 - 749 0.00 0.03 0.18 0.04 0.00 0.26 
750 - 789 0.00 0.00 0.04 0.27 0.02 0.34 
ear 790 - 850 0.00 0.00 0.00 0.02 0.08 0.10 
650 — 699 0.10 0.03 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.03 0.11 0.05 0.00 0.00 0.19 
Consistency 725 - 749 0.00 0.04 0.15 0.05 0.00 0.25 
750 - 789 0.00 0.00 0.06 0.25 0.03 0.33 
790 - 850 0.00 0.00 0.00 0.03 0.08 0.11 
650 — 699 0.12 0.02 0.00 0.00 0.00 0.14 
Decision 700 - 724 0.03 0.14 0.04 0.00 0.00 0.22 
Accuracy 725 - 749 0.00 0.04 0.20 0.04 0.00 0.28 
750 - 789 0.00 0.00 0.04 0.24 0.02 0.30 
790 - 850 0.00 0.00 0.00 0.01 0.06 0.07 
as 650 - 699 0.11 0.03 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.12 0.06 0.00 0.00 0.21 
Consistency 725 - 749 0.00 0.05 0.16 0.05 0.00 0.26 
750 - 789 0.00 0.00 0.06 0.21 0.02 0.30 
790 - 850 0.00 0.00 0.00 0.02 0.05 0.08 
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Table A.8.11 Reliability of Classification: Grade 4 Mathematics 


Full Summative Category 
eeaieeenie Level 1 Level 2 Level 3 Level 4 Level 5 arate 
650 — 699 0.11 0.02 0.00 0.00 0.00 0.12 
Decision 700 - 724 0.02 0.17 0.03 0.00 0.00 0.23 
Accuracy 725 - 749 0.00 0.04 0.21 0.04 0.00 0.28 
750 - 795 0.00 0.00 0.04 0.28 0.01 0.33 
ear 796 - 850 0.00 0.00 0.00 0.01 0.03 0.04 
650 - 699 0.10 0.03 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.03 0.14 0.05 0.00 0.00 0.22 
Consistency 725 - 749 0.00 0.05 0.17 0.05 0.00 0.27 
750 - 795 0.00 0.00 0.05 0.26 0.01 0.33 
796 - 850 0.00 0.00 0.00 0.01 0.03 0.04 
650 — 699 0.16 0.03 0.00 0.00 0.00 0.19 
Decision 700 - 724 0.03 0.20 0.04 0.00 0.00 0.28 
Accuracy 725 - 749 0.00 0.04 0.20 0.04 0.00 0.27 
750 - 795 0.00 0.00 0.03 0.20 0.01 0.24 
796 - 850 0.00 0.00 0.00 0.00 0.02 0.03 
as 650 - 699 0.15 0.04 0.00 0.00 0.00 0.20 
Decision 700 - 724 0.04 0.17 0.06 0.00 0.00 0.27 
Consistency 725 - 749 0.00 0.06 0.16 0.05 0.00 0.26 
750 - 795 0.00 0.00 0.05 0.18 0.01 0.24 
796 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
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Table A.8.12 Reliability of Classification: Grade 5 Mathematics 


Full Summative Category 
eeaieeenie Level 1 Level 2 Level 3 Level 4 Level 5 rata 
650 — 699 0.09 0.02 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.02 0.17 0.04 0.00 0.00 0.24 
Accuracy 725 - 749 0.00 0.04 0.22 0.04 0.00 0.30 
750 - 789 0.00 0.00 0.04 0.25 0.02 0.30 
ear 790 - 850 0.00 0.00 0.00 0.01 0.04 0.05 
650 - 699 0.09 0.03 0.00 0.00 0.00 0.12 
Decision 700 - 724 0.03 0.15 0.06 0.00 0.00 0.23 
Consistency 725 - 749 0.00 0.05 0.18 0.05 0.00 0.29 
750 - 789 0.00 0.00 0.06 0.22 0.02 0.30 
790 - 850 0.00 0.00 0.00 0.02 0.04 0.06 
650 — 699 0.12 0.03 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.23 0.05 0.00 0.00 0.31 
Accuracy 725 - 749 0.00 0.05 0.21 0.04 0.00 0.30 
750 - 789 0.00 0.00 0.03 0.17 0.01 0.21 
790 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
as 650 - 699 0.11 0.05 0.00 0.00 0.00 0.16 
Decision 700 - 724 0.04 0.20 0.07 0.00 0.00 0.30 
Consistency 725 - 749 0.00 0.06 0.18 0.04 0.00 0.29 
750 - 789 0.00 0.00 0.05 0.16 0.01 0.22 
790 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
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Table A.8.13 Reliability of Classification: Grade 6 Mathematics 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.12 0.02 0.00 0.00 0.00 0.14 
Decision 700 - 724 0.03 0.18 0.04 0.00 0.00 0.25 
Accuracy 725 - 749 0.00 0.04 0.21 0.04 0.00 0.29 
750 - 787 0.00 0.00 0.03 0.23 0.01 0.28 
ear 788 - 850 0.00 0.00 0.00 0.01 0.04 0.05 
650 — 699 0.12 0.03 0.00 0.00 0.00 0.15 
Decision 700 - 724 0.03 0.16 0.05 0.00 0.00 0.24 
Consistency 725 - 749 0.00 0.05 0.18 0.05 0.00 0.28 
750 - 787 0.00 0.00 0.05 0.21 0.02 0.27 
788 - 850 0.00 0.00 0.00 0.02 0.04 0.05 
650 — 699 0.09 0.02 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.02 0.20 0.04 0.00 0.00 0.26 
Accuracy 725 - 749 0.00 0.04 0.25 0.04 0.00 0.29 
750 - 787 0.00 0.00 0.04 0.25 0.02 0.31 
788 - 850 0.00 0.00 0.00 0.01 0.06 0.07 
as 650 - 699 0.08 0.03 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.03 0.15 0.05 0.00 0.00 0.23 
Consistency 725 - 749 0.00 0.06 0.17 0.05 0.00 0.28 
750 - 787 0.00 0.00 0.05 0.23 0.02 0.30 
788 - 850 0.00 0.00 0.00 0.02 0.05 0.08 
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Table A.8.14 Reliability of Classification: Grade 7 Mathematics 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 - 699 0.09 0.02 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.02 0.20 0.04 0.00 0.00 0.26 
Accuracy 725 - 749 0.00 0.04 0.25 0.04 0.00 0.34 
750 - 785 0.00 0.00 0.04 0.21 0.01 0.26 
ear 786 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 - 699 0.09 0.03 0.00 0.00 0.00 0.12 
Decision 700 - 724 0.03 0.17 0.06 0.00 0.00 0.26 
Consistency 725 - 749 0.00 0.06 0.21 0.05 0.00 0.32 
750 - 785 0.00 0.00 0.06 0.19 0.01 0.26 
786 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 — 699 0.09 0.02 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.03 0.22 0.05 0.00 0.00 0.30 
Accuracy 725 - 749 0.00 0.05 0.26 0.05 0.00 0.36 
750 - 785 0.00 0.00 0.04 0.17 0.01 0.22 
786 - 850 0.00 0.00 0.00 0.00 0.01 0.01 
as 650 - 699 0.09 0.04 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.03 0.18 0.07 0.00 0.00 0.29 
Consistency 725 - 749 0.00 0.07 0.21 0.05 0.00 0.33 
750 - 785 0.00 0.00 0.06 0.15 0.01 0.23 
786 - 850 0.00 0.00 0.00 0.01 0.01 0.02 
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Table A.8.15 Reliability of Classification: Grade 8 Mathematics 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.23 0.03 0.00 0.00 0.00 0.26 
Decision 700 - 724 0.04 0.15 0.05 0.00 0.00 0.24 
Accuracy 725 - 749 0.00 0.04 0.16 0.04 0.00 0.24 
750 - 800 0.00 0.00 0.04 0.19 0.01 0.24 
ear 801 - 850 0.00 0.00 0.00 0.00 0.01 0.02 
650 - 699 0.22 0.05 0.00 0.00 0.00 0.27 
Decision 700 - 724 0.05 0.12 0.06 0.00 0.00 0.24 
Consistency 725 - 749 0.00 0.05 0.13 0.05 0.00 0.23 
750 - 800 0.00 0.00 0.05 0.17 0.01 0.24 
801 - 850 0.00 0.00 0.00 0.01 0.01 0.02 
650 - 699 0.20 0.04 0.00 0.00 0.00 0.24 
Decision 700 - 724 0.04 0.15 0.05 0.00 0.00 0.24 
Accuracy 725 - 749 0.00 0.05 0.13 0.05 0.00 0.23 
750 - 800 0.00 0.00 0.04 0.19 0.02 0.25 
801 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
as 650 - 699 0.19 0.06 0.01 0.00 0.00 0.26 
Decision 700 - 724 0.05 0.11 0.06 0.01 0.00 0.23 
Consistency 725 - 749 0.01 0.06 0.10 0.05 0.00 0.21 
750 - 800 0.00 0.01 0.06 0.17 0.02 0.26 
801 — 850 0.00 0.00 0.00 0.02 0.02 0.04 
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Table A.8.16 Reliability of Classification: Algebra | 


Full Summative Category 
eeaie canna Level 1 Level 2 Level 3 Level 4 Level 5 rete 
650 — 699 0.11 0.02 0.00 0.00 0.00 0.13 
Decision 700 - 724 0.03 0.19 0.03 0.00 0.00 0.25 
Accuracy 725 - 749 0.00 0.04 0.18 0.04 0.00 0.26 
750 - 804 0.00 0.00 0.03 0.29 0.01 0.33 
ear 805 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 - 699 0.11 0.04 0.00 0.00 0.00 0.14 
Decision 700 - 724 0.03 0.16 0.05 0.00 0.00 0.24 
Consistency 725 - 749 0.00 0.05 0.15 0.05 0.00 0.25 
750 - 804 0.00 0.00 0.05 0.27 0.01 0.33 
805 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 - 699 0.13 0.03 0.00 0.00 0.00 0.17 
Decision 700 - 724 0.03 0.20 0.04 0.00 0.00 0.27 
Accuracy 725 - 749 0.00 0.05 0.14 0.04 0.00 0.24 
750 - 804 0.00 0.00 0.03 0.27 0.01 0.31 
805 - 850 0.00 0.00 0.00 0.01 0.02 0.02 
as 650 - 699 0.13 0.05 0.00 0.00 0.00 0.18 
Decision 700 - 724 0.04 0.16 0.05 0.00 0.00 0.25 
Consistency 725 - 749 0.00 0.06 0.11 0.06 0.00 0.24 
750 - 804 0.00 0.00 0.04 0.25 0.01 0.31 
805 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
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Table A.8.17 Reliability of Classification: Geometry 


Full Summative Category 
eeaieeenve Level 1 Level 2 Level 3 Level 4 Level 5 rete 
650 — 699 0.07 0.02 0.00 0.00 0.00 0.09 
Decision 700 - 724 0.02 0.22 0.03 0.00 0.00 0.27 
Accuracy 725 - 749 0.00 0.04 0.25 0.04 0.00 0.33 
750 - 782 0.00 0.00 0.03 0.23 0.01 0.28 
ear 783 - 850 0.00 0.00 0.00 0.01 0.03 0.04 
650 — 699 0.07 0.03 0.00 0.00 0.00 0.10 
Decision 700 - 724 0.02 0.20 0.05 0.00 0.00 0.27 
Consistency 725 - 749 0.00 0.05 0.22 0.05 0.00 0.32 
750 - 782 0.00 0.00 0.03 0.23 0.01 0.28 
783 - 850 0.00 0.00 0.00 0.01 0.03 0.04 
650 — 699 0.09 0.02 0.00 0.00 0.00 0.10 
Decision 700 - 724 0.02 0.24 0.04 0.00 0.00 0.29 
Accuracy 725 - 749 0.00 0.04 0.27 0.04 0.00 0.35 
750 - 782 0.00 0.00 0.03 0.21 0.01 0.25 
783 - 850 0.00 0.00 0.00 0.00 0.00 0.00 
as 650 - 699 0.08 0.03 0.00 0.00 0.00 0.11 
Decision 700 - 724 0.03 0.21 0.05 0.00 0.00 0.29 
Consistency 725 - 749 0.00 0.06 0.23 0.05 0.00 0.34 
750 - 782 0.00 0.00 0.04 0.20 0.00 0.25 
783 - 850 0.00 0.00 0.00 0.00 0.00 0.01 
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Table A.8.18 Reliability of Classification: Algebra II 


Full Summative Category 
Level 1 Level 2 Level 3 Level 4 Level 5 

Scale Score Total 
650 — 699 0.29 0.04 0.00 0.00 0.00 0.33 
Decision 700 - 724 0.04 0.15 0.04 0.00 0.00 0.23 
Accuracy 725 - 749 0.00 0.04 0.11 0.04 0.00 0.19 
750 - 807 0.00 0.00 0.03 0.20 0.01 0.23 
ear 808 - 850 0.00 0.00 0.00 0.00 0.02 0.02 
650 — 699 0.28 0.05 0.00 0.00 0.00 0.33 
Decision 700 - 724 0.05 0.12 0.05 0.00 0.00 0.22 
Consistency 725 - 749 0.00 0.05 0.09 0.04 0.00 0.18 
750 - 807 0.00 0.00 0.04 0.18 0.01 0.24 
808 - 850 0.00 0.00 0.00 0.01 0.02 0.03 
650 - 699 0.19 0.02 0.00 0.00 0.00 0.21 
Decision 700 - 724 0.03 0.12 0.04 0.00 0.00 0.20 
Accuracy 725 - 749 0.00 0.04 0.16 0.04 0.00 0.23 
750 - 807 0.00 0.00 0.04 0.28 0.01 0.33 
808 - 850 0.00 0.00 0.00 0.01 0.02 0.02 
as 650 - 699 0.18 0.04 0.00 0.00 0.00 0.22 
Decision 700 - 724 0.04 0.10 0.06 0.00 0.00 0.20 
Consistency 725 - 749 0.00 0.04 0.12 0.05 0.00 0.22 
750 - 807 0.00 0.01 0.06 0.25 0.01 0.33 
808 - 850 0.00 0.00 0.00 0.01 0.02 0.01 
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Table A.8.19 Reliability of Classification: Integrated Mathematics | 


Full Summative Category 
eeaieeenve Level 1 Level 2 Level 3 Level 4 Level 5 Teel 
650 — 699 0.15 0.03 0.00 0.00 0.00 0.18 
Decision 700 - 724 0.04 0.20 0.05 0.00 0.00 0.28 
Accuracy 725 - 749 0.00 0.04 0.19 0.04 0.00 0.27 
750 - 798 0.00 0.00 0.03 0.20 0.01 0.24 
799 - 850 0.00 0.00 0.00 0.00 0.02 0.03 
ia 650 - 699 0.14 0.05 0.00 0.00 0.00 0.19 
Decision 700 - 724 0.05 0.16 0.06 0.00 0.00 0.27 
Consistency 725 - 749 0.00 0.06 0.15 0.05 0.00 0.26 
750 - 798 0.00 0.00 0.05 0.19 0.01 0.25 
799 - 850 0.00 0.00 0.00 0.01 0.02 0.03 


March 27, 2018 Page 316 


PARCC 2017 Technical Report 


Table A.8.20 Reliability of Classification: Integrated Mathematics II 


Full Summative Category 
eeaieeenve Level 1 Level 2 Level 3 Level 4 Level 5 retell 
650 — 699 0.05 0.01 0.00 0.00 0.00 0.05 
Decision 700 - 724 0.02 0.11 0.03 0.00 0.00 0.16 
Accuracy 725 - 749 0.00 0.04 0.20 0.05 0.00 0.29 
750 - 784 0.00 0.00 0.05 0.31 0.03 0.39 
785 - 850 0.00 0.00 0.00 0.02 0.07 0.10 
ia 650 - 699 0.05 0.02 0.00 0.00 0.00 0.07 
Decision 700 - 724 0.02 0.09 0.05 0.00 0.00 0.17 
Consistency 725 - 749 0.00 0.04 0.16 0.07 0.000 0.28 
750 - 784 0.00 0.00 0.07 0.27 0.03 0.37 
785 - 850 0.00 0.00 0.00 0.04 0.07 0.12 
650 — 699 0.23 0.07 0.00 0.00 0.00 0.31 
Decision 700 - 724 0.06 0.28 0.04 0.00 0.00 0.38 
Accuracy 725 - 749 0.00 0.06 0.10 0.03 0.00 0.19 
750 - 784 0.00 0.00 0.01 0.09 0.01 0.11 
785 - 850 0.00 0.00 0.00 0.00 0.01 0.02 
as 650 - 699 0.22 0.10 0.00 0.00 0.00 0.32 
Decision 700 - 724 0.07 0.22 0.04 0.00 0.00 0.34 
Consistency 725 - 749 0.00 0.08 0.08 0.04 0.00 0.20 
750 - 784 0.00 0.00 0.02 0.08 0.01 0.12 
785 - 850 0.00 0.00 0.00 0.01 0.01 0.02 
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Table A.8.21 Reliability of Classification: Integrated Mathematics III 


Full Summative Category 
eeaieeenve Level 1 Level 2 Level 3 Level 4 Level 5 Teel 
650 — 699 0.35 0.02 0.00 0.00 0.00 0.38 
Decision 700 - 724 0.04 0.09 0.03 0.00 0.00 0.16 
Accuracy 725 - 749 0.00 0.03 0.09 0.04 0.00 0.16 
750 - 803 0.00 0.00 0.03 0.23 0.01 0.27 
804 - 850 0.00 0.00 0.00 0.01 0.02 0.02 
ia 650 - 699 0.34 0.04 0.00 0.00 0.00 0.38 
Decision 700 - 724 0.05 0.07 0.04 0.01 0.00 0.17 
Consistency 725 - 749 0.01 0.03 0.07 0.05 0.00 0.16 
750 - 803 0.00 0.00 0.04 0.21 0.01 0.26 
804 - 850 0.00 0.00 0.00 0.02 0.01 0.03 
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Appendix 10.1: IRT Results for Spring 2017 English Language Arts/Literacy (ELA/L) 


Table A.10.1 CBT IRT Summary Parameter Estimates for All Items for ELA/L by Grade 


Item No. of No. of b Estimates Summary a Estimates Summary 
Mode Grade Grouping Score Points Items Mean SD Min Max Mean SD Min Max 
CBT E03 All Items 180 82 0.47 1.05 -1.40 3.44 0.55 0.2 0.11 1.00 
CBT E03 Reading 132 66 0.15 0.91 -1.40 3.44 0.49 0.16 0.11 0.81 
CBT E03 Writing 48 16 1.77 0.35 1.16 2.22 0.83 0.11 0.51 1.00 
CBT E04 All Items 193 86 0.20 1.43 -6.48 4.99 0.49 0.22 0.13 0.97 
CBT E04 Reading 140 70 -0.04 1.48 -6.48 4.99 0.41 0.14 0.13 0.77 
CBT E04 Writing 53 16 1.24 0.40 0.53 1.98 0.85 0.08 0.68 0.97 
CBT EOS All tems 192 85 0.65 1.21 -6.27 3.56 0.47 0.23 0.13 1.00 
CBT E05 Reading 138 69 0.45 1.22 -6.27 3.56 0.38 0.13 0.13 0.65 
CBT E05 Writing 54 16 1.50 0.67 0.70 2.69 0.88 0.09 0.64 1.00 
CBT E06 All Items 233 106 0.53 0.87 -0.97 4.45 0.48 0.19 0.15 0.96 
CBT E06 Reading 184 92 0.44 0.89 -0.97 4.45 0.43 0.14 0.15 0.90 
CBT E06 Writing 49 14 1.14 0.42 0.43 1.97 0.81 0.09 0.66 0.96 
CBT EO7 All Items 217 98 0.35 0.89 -2.17 3.82 0.48 0.23 0.14 1.11 
CBT EO7 Reading 168 84 0.29 0.94 -2.17 3.82 0.40 0.14 0.14 0.74 
CBT EO7 Writing 49 14 0.71 0.41 0.18 1.47 0.91 0.14 0.66 1.11 
CBT E08 All Items 232 104 0.27 0.95 -2.39 3.30 0.46 0.26 0.09 1.10 
CBT E08 Reading 176 88 0.21 1.01 -2.39 3.30 0.37 0.14 0.09 0.74 
CBT E08 Writing 56 16 0.59 0.43 -0.20 1.32 0.97 0.11 0.67 1.10 
CBT E09 All Items 210 96 0.58 0.83 -1.04 3.59 0.47 0.28 0.10 1.40 
CBT E09 Reading 168 84 0.54 0.88 -1.04 3.59 0.37 0.13 0.10 0.76 
CBT E09 Writing 42 12 0.83 0.29 0.37 1.33 1.14 0.13 0.96 1.40 
CBT E10 All Items 232 104 0.69 0.80 -1.27 3.06 0.48 0.23 0.16 1.09 
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Item No. of No. of b Estimates Summary a Estimates Summary 
Mode Grade Grouping Score Points Items Mean SD Min Max Mean SD Min Max 
CBT E10 Reading 176 88 0.67 0.85 -1.27 3.06 0.40 0.14 0.16 0.93 
CBT E10 Writing 56 16 0.79 0.33 0.27 1.35 0.92 0.10 0.77 1.09 
CBT E11 All Items 236 106 0.91 0.82 -0.54 3.77 0.48 0.19 0.14 0.91 
CBT E11 Reading 180 90 0.87 0.87 -0.54 3.77 0.43 0.16 0.14 0.88 
CBT E11 Writing 56 16 1.13 0.33 0.61 1.88 0.74 0.07 0.65 0.91 
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Table A.10.2 PBT IRT Summary Parameter Estimates for All Items for ELA/L by Grade 


No. of b Estimates Summary a Estimates Summary 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
PBT E03 All Items 70 32 0.82 0.96 -0.90 3.35 0.50 0.18 0.12 0.82 
PBT E03 Reading 52 26 0.68 1.00 -0.90 3.35 0.45 0.14 0.12 0.74 
PBT E03 Writing 18 6 1.44 0.39 1.01 2.14 0.74 0.11 0.51 0.82 
PBT E04 All Items 76 34 0.51 1.09 -1.08 4.96 0.48 0.24 0.17 0.99 
PBT E04 Reading 56 28 0.45 1.17 -1.08 4.96 0.39 0.14 0.17 0.77 
PBT E04 Writing 20 6 0.83 0.45 0.39 1.47 0.90 0.08 0.80 0.99 
PBT E05 All Items 76 34 1.03 0.83 -0.97 2.99 0.43 0.22 0.13 0.94 
PBT E05 Reading 56 28 1.04 0.90 -0.97 2.99 0.35 0.13 0.13 0.67 
PBT E05 Writing 20 6 0.98 0.49 0.54 1.69 0.82 0.11 0.63 0.94 
PBT E06 All Items 89 40 0.51 0.78 -0.74 2.37 0.44 0.19 0.15 0.96 
PBT E06 Reading 68 34 0.45 0.82 -0.74 2.37 0.39 0.15 0.15 0.68 
PBT E06 Writing 21 6 0.85 0.38 0.41 1.39 0.73 0.16 0.53 0.96 
PBT E07 All Items 89 40 0.38 0.80 -1.38 2.15 0.38 0.16 0.19 0.84 
PBT E07 Reading 68 34 0.32 0.84 -1.38 2.15 0.33 0.08 0.19 0.53 
PBT E07 Writing 21 6 0.67 0.41 -0.03 1.15 0.71 0.10 0.58 0.84 
PBT E08 All Items 89 40 0.35 0.78 -1.39 1.90 0.43 0.19 0.10 0.84 
PBT E08 Reading 68 34 0.38 0.83 -1.39 1.90 0.38 0.16 0.10 0.77 
PBT E08 Writing 21 6 0.14 0.39 -0.32 0.57 0.69 0.11 0.52 0.84 
PBT E09 All Items 89 40 0.58 0.77 -1.07 2.22 0.44 0.21 0.18 0.96 
PBT E09 Reading 68 34 0.50 0.79 -1.07 2.22 0.37 0.13 0.18 0.81 
PBT E09 Writing 21 6 1.05 0.42 0.51 1.62 0.83 0.11 0.70 0.96 
PBT E10 All Items 89 40 0.48 0.73 -1.05 1.88 0.51 0.19 0.18 0.90 
PBT E10 Reading 68 34 0.44 0.77 -1.05 1.88 0.46 0.15 0.18 0.83 
PBT E10 Writing 21 6 0.75 0.40 0.23 1.23 0.80 0.07 0.68 0.90 
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No. of b Estimates Summary a Estimates Summary 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
PBT E11 All Items 89 40 1.35 1.00 -0.58 3.75 0.41 0.21 0.15 0.90 
PBT E11 Reading 68 34 1.39 1.07 -0.58 3.75 0.36 0.19 0.15 0.90 
PBT E11 Writing 21 6 1.12 0.39 0.60 1.53 0.67 0.06 0.57 0.72 
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Table A.10.3 CBT IRT Standard Errors of Parameter Estimates for All Items for ELA/L by Grade 


No. of SE of b Estimates SE of a Estimates 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
CBT E03 All Items 180 82 0.009 0.007 0.004 0.053 0.004 0.002 0.002 0.010 
CBT E03 Reading 132 66 0.008 0.007 0.004 0.053 0.004 0.001 0.002 0.007 
CBT E03 Writing 48 16 0.014 0.004 0.008 0.020 0.008 0.001 0.006 0.010 
CBT E04 All Items 193 86 0.013 0.022 0.004 0.163 0.004 0.002 0.002 0.010 
CBT E04 Reading 140 70 0.014 0.024 0.004 0.163 0.003 0.001 0.002 0.007 
CBT E04 Writing 53 16 0.011 0.005 0.004 0.026 0.008 0.001 0.006 0.010 
CBT E05 All Items 192 85 0.012 0.017 0.004 0.140 0.004 0.002 0.002 0.010 
CBT E05 Reading 138 69 0.012 0.018 0.004 0.140 0.003 0.001 0.002 0.005 
CBT E05 Writing 54 16 0.015 0.011 0.004 0.039 0.008 0.001 0.004 0.010 
CBT E06 All Items 233 106 0.010 0.010 0.004 0.093 0.004 0.002 0.002 0.010 
CBT E06 Reading 184 92 0.010 0.010 0.004 0.093 0.004 0.001 0.002 0.007 
CBT E06 Writing 49 14 0.010 0.005 0.004 0.023 0.008 0.001 0.005 0.010 
CBT EO7 All Items 217 98 0.010 0.008 0.004 0.067 0.005 0.002 0.002 0.012 
CBT EO7 Reading 168 84 0.011 0.008 0.005 0.067 0.004 0.001 0.002 0.006 
CBT EO7 Writing 49 14 0.007 0.003 0.004 0.013 0.009 0.002 0.006 0.012 
CBT E08 All Items 232 104 0.012 0.010 0.004 0.075 0.005 0.003 0.002 0.013 
CBT E08 Reading 176 88 0.013 0.011 0.005 0.075 0.004 0.001 0.002 0.007 
CBT E08 Writing 56 16 0.008 0.004 0.004 0.018 0.010 0.002 0.007 0.013 
CBT E09 All Items 210 96 0.012 0.011 0.004 0.067 0.005 0.003 0.002 0.017 
CBT E09 Reading 168 84 0.013 0.011 0.005 0.067 0.004 0.001 0.002 0.006 
CBT E09 Writing 42 12 0.006 0.002 0.004 0.010 0.012 0.002 0.009 0.017 
CBT E10 All Items 232 104 0.015 0.010 0.005 0.080 0.006 0.003 0.003 0.016 
CBT E10 Reading 176 88 0.016 0.011 0.006 0.080 0.005 0.001 0.003 0.010 
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No. of SE of b Estimates SE of a Estimates 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
CBT E10 Writing 56 16 0.010 0.003 0.005 0.015 0.013 0.002 0.010 0.016 
CBT E11 All Items 236 106 0.019 0.015 0.007 0.099 0.007 0.003 0.003 0.016 
CBT E11 Reading 180 90 0.020 0.016 0.007 0.099 0.006 0.002 0.003 0.011 
CBT E11 Writing 56 16 0.015 0.005 0.009 0.031 0.012 0.002 0.009 0.016 
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No. of G? Qi 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
CBT E03 All Items 180 82 2628.5 2558.7 182.3 11676.5 2524.8 2468.9 178.9 10234.5 
CBT E03 Reading 132 66 3012.4 2714.2 182.3 11676.5 2889.7 2622.2 178.9 10234.5 
CBT E03 Writing 48 16 1044.9 327.1 526.7 1767.3 1019.5 342 517.9 1755.5 
CBT E04 All Items 193 86 2566 2756.7 391.7 20517.1 2488.4 2633.2 373.6 17734.4 
CBT E04 Reading 140 70 2820.5 2994 391.7 20517.1 2730.9 2858.1 373.6 17734.4 
CBT E04 Writing 53 16 1452.7 450.4 1007.7 2423.1 1427.5 488.6 945.6 2420.5 
CBT E05 All Items 192 85 3010.8 3148.6 132.3 20149.4 9706.6 62643.1 129.6 579715.1 
CBT E05 Reading 138 69 3325.4 3405 132.3 20149.4 3207.8 3458.5 129.6 22452.8 
CBT E05 Writing 54 16 1654 732.5 1023 3810.2 37732.6 144530.1 1014.7 579715.1 
CBT E06 All Items 233 106 2033.8 1846.4 227.5 12312.5 1964.5 1828.4 222.5 12034 
CBT E06 Reading 184 92 2096.1 1957.8 227.5 12312.5 2042.6 1937.7 222.5 12034 
CBT E06 Writing 49 14 1624.7 704.7 865.8 3583.5 1450.8 625.7 786.1 3191.6 
CBT E07 All Items 217 98 2007 2081.5 189.3 11308.1 1979.5 2096.7 180.7 10699.5 
CBT E07 Reading 168 84 2069.8 2225.5 189.3 11308.1 2060.9 2242 180.7 10699.5 
CBT E07 Writing 49 14 1629.8 726.7 896.5 3542.2 1491.1 641.2 815.3 3109.7 
CBT E08 All Items 232 104 1922.6 1726.3 164.2 9222.1 1853.3 1678.6 152.6 8390.7 
CBT E08 Reading 176 88 1978.9 1852.7 164.2 9222.1 1926.5 1799.7 152.6 8390.7 
CBT E08 Writing 56 16 1612.9 658.6 922.3 3385.1 1450.6 599.1 819.6 3049.7 
CBT E09 All Items 210 96 1683.8 1348.7 150.2 5455.1 1671.9 1443.7 148.7 7125.8 
CBT E09 Reading 168 84 1708.7 1420 150.2 5455.1 1706.4 1516.9 148.7 7125.8 
CBT E09 Writing 42 12 1508.9 675.2 807.7 2529.8 1430.6 751.9 693.6 3132.4 
CBT E10 All Items 232 104 1154.7 1286 58.1 11355.6 1127.9 1500.8 57.2 14014.8 
CBT E10 Reading 176 88 1159.9 1387.4 58.1 11355.6 1156.9 1623.6 57.2 14014.8 


March 27, 2018 


Page 325 


PARCC 


2017 Technical Report 


No. of CG 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
CBT E10 Writing 56 16 1126.3 436.8 682.7 2219.3 968 380.8 604.1 1927.5 
CBT E11 All Items 236 106 747.9 700.2 74.9 5082.1 700.3 665 75.6 4584.2 
CBT E11 Reading 180 90 713 740.8 74.9 5082.1 678.8 708.7 75.6 4584.2 
CBT E11 Writing 56 16 944.2 356.8 625 1986.5 821.4 311.7 541.3 1711.7 
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Appendix 10.2: IRT Results for Spring 2017 Mathematics 


Table A.10.5 CBT IRT Summary Parameter Estimates for All Items for Mathematics by Grade/Subject 


No. of b Estimates Summary a Estimates Summary 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
CBT M03 All Items 172 113 -0.01 1.13 -3.00 2.81 0.77 0.25 0.22 1.42 
CBT M03 SSMC 24 24 -0.28 1.08 -2.56 1.99 0.69 0.28 0.26 1.31 
CBT M03 CR 148 89 0.07 1.14 -3.00 2.81 0.80 0.24 0.22 1.42 
CBT M03 Type | 106 96 -0.15 1.13 -3.00 2.35 0.81 0.25 0.22 1.42 
CBT M03 Type Il 36 10 0.51 0.87 -0.16 2.81 0.58 0.15 0.40 0.82 
CBT M03 Type III 30 7 1.16 0.23 0.78 1.46 0.53 0.14 0.30 0.76 
CBT M04 All Items 155 99 0.13 0.93 -1.98 2.10 0.76 0.19 0.29 1.23 
CBT M04 SSMC 11 11 -0.82 0.75 -1.98 0.65 0.70 0.09 0.49 0.80 
CBT M04 CR 144 88 0.25 0.88 -1.78 2.10 0.77 0.20 0.29 1.23 
CBT M04 Type | 101 84 -0.04 0.86 -1.98 2.10 0.79 0.19 0.34 1.23 
CBT M04 Type Il 30 9 1.02 0.86 -0.94 2.06 0.64 0.17 0.29 0.84 
CBT M04 Type III 24 6 1.11 0.61 0.26 2.10 0.59 0.19 0.33 0.85 
CBT MO5 All Items 164 103 0.22 1.08 -2.44 2.61 0.68 0.25 0.22 1.45 
CBT MO5 SSMC 14 14 -0.25 1.02 -1.55 1.56 0.71 0.32 0.22 1.36 
CBT MO5 CR 150 89 0.29 1.08 -2.44 2.61 0.67 0.23 0.23 1.45 
CBT MO5 Type | 108 87 0.05 1.08 -2.44 2.61 0.69 0.26 0.22 1.45 
CBT MOS Type Il 32 9 1.06 0.57 -0.21 1.71 0.53 0.13 0.30 0.68 
CBT MOo5 Type Ill 24 7 1.16 0.52 0.54 2.14 0.68 0.17 0.42 0.89 
CBT M06 All Items 163 95 0.28 1.10 -2.98 2.46 0.72 0.25 0.26 1.38 
CBT M06 SSMC 16 16 -0.22 1.23 -2.70 1.69 0.56 0.20 0.26 1.03 
CBT M06 CR 147 79 0.38 1.05 -2.98 2.46 0.75 0.24 0.29 1.38 
CBT M06 Type | 99 78 0.12 1.12 -2.98 2.46 0.76 0.25 0.26 1.38 
CBT M06 Type Il 34 10 1.01 0.68 -0.15 2.37 0.51 0.14 0.29 0.76 
CBT M06 Type III 30 7 1.00 0.65 0.07 1.73 0.58 0.09 0.42 0.72 
CBT M07 All Items 163 97 1.07 1.08 -1.76 3.47 0.70 0.28 0.24 1.71 
CBT M07 SSMC 16 16 -0.05 0.91 -1.76 1.55 0.56 0.21 0.24 0.87 
CBT M07 CR 147 81 1.29 0.96 -1.04 3.47 0.73 0.28 0.29 1.71 
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No. of b Estimates Summary a Estimates Summary 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
CBT M07 Type | 106 81 1.01 1.11 -1.76 3.47 0.72 0.29 0.24 1.71 
CBT M07 Type Il 36 10 1.32 0.63 0.09 2.28 0.60 0.14 0.35 0.82 
CBT M07 Type Ill 21 6 1.44 1.22 0.14 3.16 0.59 0.15 0.41 0.78 
CBT M08 All Items 166 91 1.10 1.07 -1.80 3.92 0.65 0.26 0.23 1.35 
CBT M08 SSMC 24 24 0.32 1.01 -1.80 1.99 0.48 0.16 0.23 0.82 
CBT M08 CR 142 67 1.38 0.94 -0.73 3.92 0.70 0.26 0.25 1.35 
CBT M08 Type | 100 74 0.95 1.09 -1.80 3.92 0.67 0.27 0.23 1.35 
CBT MO8 Type Il 36 10 1.82 0.54 0.74 2.50 0.53 0.17 0.31 0.76 
CBT M08 Type III 30 7 1.71 0.82 0.12 2.31 0.52 0.09 0.41 0.63 
CBT Al All Items 191 104 1.53 0.92 -0.73 5.30 0.67 0.28 0.23 1.34 
CBT Al SSMC 22 22 0.98 0.86 -0.73 2.74 0.46 0.15 0.25 0.71 
CBT Al CR 169 82 1.68 0.88 -0.20 5.30 0.73 0.28 0.23 1.34 
CBT Al Type | 115 85 1.52 0.95 -0.73 5.30 0.68 0.31 0.23 1.34 
CBT Al Type Il 28 8 1.69 0.74 0.20 2.54 0.65 0.14 0.48 0.88 
CBT Al Type III 48 11 1.49 0.79 0.39 2.59 0.60 0.10 0.48 0.79 
CBT GO All Items 206 110 1.14 0.98 -1.77 3.88 0.77 0.33 0.16 1.86 
CBT GO SSMC 15 15 0.09 0.89 -1.77 1.60 0.51 0.19 0.16 0.83 
CBT GO CR 191 95 1.30 0.89 -0.36 3.88 0.82 0.33 0.19 1.86 
CBT GO Type | 123 90 0.98 1.01 -1.77 3.88 0.79 0.35 0.16 1.86 
CBT GO Type Il 29 8 1.80 0.33 1.32 2.28 0.75 0.19 0.45 1.11 
CBT GO Type III 54 12 1.85 0.47 0.99 2.52 0.72 0.21 0.39 1.06 
CBT A2 All Items 194 102 1.56 0.91 -0.51 4.21 0.70 0.33 0.18 1.56 
CBT A2 SSMC 17 17 0.81 0.88 -0.51 2.26 0.52 0.18 0.20 0.89 
CBT A2 CR 177 85 1.71 0.84 -0.24 4.21 0.73 0.34 0.18 1.56 
CBT A2 Type | 116 82 1.41 0.91 -0.51 4.21 0.70 0.35 0.18 1.56 
CBT A2 Type Il 36 10 2.10 0.62 0.72 3.14 0.69 0.25 0.33 1.04 
CBT A2 Type III 42 10 2.28 0.56 1.40 2.96 0.66 0.25 0.31 1.01 
CBT M1 All Items 81 42 1.15 1.07 -0.95 3.94 0.64 0.22 0.18 1.28 
CBT M1 SSMC 11 11 0.28 0.93 -0.95 2.25 0.52 0.24 0.22 0.92 
CBT M1 CR 70 31 1.46 0.95 -0.19 3.94 0.68 0.21 0.18 1.28 
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No. of b Estimates Summary a Estimates Summary 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
CBT M1 Type | 49 34 0.97 0.98 -0.95 3.08 0.65 0.24 0.18 1.28 
CBT M1 Type Il 14 4 1.55 1.63 0.28 3.94 0.62 0.14 0.46 0.78 
CBT M1 Type Ill 18 4 2.28 0.44 1.86 2.85 0.55 0.10 0.49 0.70 
CBT M2 All Items 80 41 1.90 1.45 -0.74 5.99 0.53 0.27 0.12 1.18 
CBT M2 SSMC 14 14 1.01 1.21 -0.74 3.24 0.37 0.13 0.12 0.56 
CBT M2 CR 66 27 2.36 1.37 0.04 5.99 0.62 0.29 0.17 1.18 
CBT M2 Type | 48 33 1.81 1.53 -0.74 5.99 0.52 0.29 0.12 1.18 
CBT M2 Type Il 14 4 2.87 0.82 2.04 3.76 0.56 0.11 0.43 0.68 
CBT M2 Type III 18 4 1.65 0.98 0.20 2.27 0.64 0.16 0.47 0.79 
CBT M3 All Items 81 40 1.27 1.08 -2.27 4.28 0.59 0.27 0.16 1.27 
CBT M3 SSMC 12 12 0.84 1.71 -2.27 4.28 0.44 0.15 0.23 0.66 
CBT M3 CR 69 28 1.46 0.61 -0.04 2.79 0.66 0.29 0.16 1.27 
CBT M3 Type | 49 32 1.20 1.17 -2.27 4.28 0.58 0.29 0.16 1.27 
CBT M3 Type Il 14 4 1.58 0.61 0.96 2.20 0.74 0.08 0.63 0.83 
CBT M3 Type III 18 4 1.57 0.57 0.90 2.11 0.53 0.20 0.30 0.76 


Note: M03 through M08 = mathematics grades 3 through 8, Al = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, M2 = Integrated 
Mathematics II], M3 = Integrated Mathematics III. 
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Table A.10.6 PBT IRT Summary Parameter Estimates for All Items for Mathematics by Grade/Subject 


No. of b Estimates Summary a Estimates Summary 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
PBT M03 All Items 66 43 -0.27 1.04 -2.44 2.08 0.74 0.26 0.29 1.42 
PBT M03 SSMC 14 14 -0.73 1.10 -2.44 0.69 0.72 0.15 0.41 0.92 
PBT M03 CR 52 29 -0.05 0.95 -2.16 2.08 0.74 0.30 0.29 1.42 
PBT M03 Type | 40 36 -0.43 1.04 -2.44 2.08 0.79 0.24 0.29 1.42 
PBT M03 Type Il 14 4 0.54 0.68 -0.32 1.35 0.43 0.09 0.37 0.56 
PBT M03 Type Ill 12 3 0.56 0.42 0.19 1.02 0.49 0.11 0.37 0.57 
PBT M04 All Items 66 40 -0.18 1.12 -2.11 2.10 0.75 0.23 0.42 1.28 
PBT M04 SSMC 10 10 -0.76 1.22 -2.11 1.44 0.74 0.25 0.43 1.19 
PBT M04 CR 56 30 0.01 1.03 -1.83 2.10 0.75 0.23 0.42 1.28 
PBT M04 Type | 40 33 -0.44 1.03 -2.11 1.53 0.78 0.23 0.43 1.28 
PBT M04 Type Il 14 4 1.25 0.68 0.65 2.10 0.68 0.15 0.50 0.86 
PBT M04 Type Ill 12 3 0.71 0.60 0.26 1.39 0.53 0.11 0.42 0.64 
PBT M05 All Items 66 40 0.00 1.16 -2.27 2.28 0.71 0.23 0.40 1.36 
PBT MO5 SSMC 15 15 -0.78 1.01 -2.27 1.14 0.73 0.27 0.40 1.36 
PBT MO5 CR 51 25 0.47 0.98 -1.49 2.28 0.70 0.21 0.40 1.27 
PBT MO5 Type | 40 33 -0.24 1.10 -2.27 2.28 0.73 0.25 0.40 1.36 
PBT MO5 Type Il 14 4 0.88 0.70 -0.03 1.47 0.58 0.09 0.48 0.68 
PBT MOo5 Type Ill 12 3 1.49 0.47 0.97 1.89 0.63 0.19 0.42 0.78 
PBT M06 All Items 66 38 0.27 1.32 -3.54 3.59 0.64 0.24 0.23 1.40 
PBT M06 SSMC 15 15 -0.41 1.46 -3.54 1.64 0.59 0.23 0.23 1.09 
PBT M06 CR 51 23 0.71 1.03 -0.89 3.59 0.68 0.24 0.30 1.40 
PBT M06 Type | 40 31 0.14 1.41 -3.54 3.59 0.67 0.26 0.23 1.40 
PBT M06 Type Il 14 4 0.61 0.54 -0.15 1.13 0.53 0.11 0.38 0.65 
PBT M06 Type III 12 3 1.19 0.61 0.48 1.57 0.54 0.07 0.47 0.61 
PBT M07 All Items 66 38 0.93 1.20 -1.66 3.06 0.61 0.21 0.34 1.06 
PBT M07 SSMC 14 14 -0.09 1.06 -1.66 1.79 0.51 0.14 0.34 0.79 
PBT M07 CR 52 24 1.53 0.81 -0.17 3.06 0.67 0.22 0.34 1.06 
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No. of b Estimates Summary a Estimates Summary 
Item Score No. of 
Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
PBT M07 Type | 40 31 0.82 1.27 -1.66 3.06 0.62 0.22 0.34 1.06 
PBT M07 Type Il 14 4 1.48 0.48 0.89 2.00 0.64 0.06 0.55 0.69 
PBT M07 Type Ill 12 3 1.36 1.08 0.14 2.22 0.54 0.12 0.45 0.68 
PBT M08 All Items 66 36 1.04 1.17 -1.17 3.22 0.56 0.24 0.18 1.23 
PBT M08 SSMC 14 14 0.60 1.28 -1.17 2.44 0.45 0.16 0.18 0.64 
PBT M08 CR 52 22 1.32 1.02 -0.48 3.22 0.64 0.25 0.27 1.23 
PBT M08 Type | 40 29 0.87 1.13 -1.17 2.58 0.57 0.25 0.18 1.23 
PBT M08 Type Il 14 4 2.09 0.98 1.11 3.22 0.56 0.25 0.27 0.87 
PBT M08 Type Ill 12 3 1.29 1.34 -0.25 2.09 0.50 0.03 0.48 0.53 
PBT Al All Items 81 42 1.05 1.14 -1.16 3.27 0.54 0.20 0.17 1.01 
PBT Al SSMC 17 17 0.25 0.92 -1.16 2.41 0.47 0.17 0.17 0.73 
PBT Al CR 64 25 1.59 0.96 -0.77 3.27 0.59 0.21 0.20 1.01 
PBT Al Type | 49 34 0.87 1.12 -1.16 3.05 0.51 0.21 0.17 1.01 
PBT Al Type Il 14 4 2.18 0.90 1.11 3.27 0.64 0.07 0.59 0.75 
PBT Al Type Ill 18 4 1.45 1.03 0.10 2.47 0.66 0.19 0.48 0.92 
PBT GO All Items 81 43 1.02 1.10 -1.10 3.60 0.71 0.32 0.24 1.41 
PBT GO SSMC 16 16 0.48 1.39 -1.10 3.60 0.51 0.23 0.24 0.98 
PBT GO CR 65 27 1.33 0.75 -0.17 2.54 0.83 0.31 0.30 1.41 
PBT GO Type | 49 35 0.82 1.13 -1.10 3.60 0.71 0.34 0.24 1.41 
PBT GO Type Il 14 4 2.01 0.26 1.72 2.30 0.78 0.25 0.51 1.11 
PBT GO Type Ill 18 4 1.69 0.37 1.39 2.22 0.68 0.13 0.55 0.82 
PBT A2 All Items 81 41 1.75 1.87 -1.39 9.06 0.61 0.29 0.12 1.41 
PBT A2 SSMC 11 11 1.72 2.89 -1.39 9.06 0.49 0.30 0.12 1.12 
PBT A2 CR 70 30 1.76 1.39 -0.81 6.66 0.65 0.28 0.24 1.41 
PBT A2 Type | 49 33 1.70 2.07 -1.39 9.06 0.59 0.30 0.12 1.41 
PBT A2 Type Il 14 4 2.00 0.67 1.36 2.95 0.78 0.24 0.50 1.04 
PBT A2 Type Ill 18 4 1.88 0.47 1.30 2.33 0.56 0.24 0.28 0.80 
PBT M1 All Items 81 42 1.22 1.31 -1.25 5.41 0.59 0.28 0.00 1.36 
PBT M1 SSMC 16 16 0.31 0.93 -1.25 2.38 0.51 0.27 0.17 1.24 
PBT M1 CR 65 26 1.78 1.19 -0.11 5.41 0.65 0.28 0.00 1.36 
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No. of b Estimates Summary a Estimates Summary 
Item Score No. of 

Mode Grade Grouping Points Items Mean SD Min Max Mean SD Min Max 
PBT M1 Type | 49 34 0.95 1.28 -1.25 5.41 0.57 0.29 0.00 1.36 
PBT M1 Type Il 14 4 2.34 0.84 1.12 2.99 0.70 0.17 0.46 0.86 
PBT M1 Type Ill 18 4 2.37 0.58 1.70 2.86 0.73 0.25 0.47 1.08 
PBT M2 All 80 42 1.67 1.14 -0.5 4.87 0.52 0.24 0.18 1.15 
PBT M2 SSMC 19 19 1.11 0.91 -0.17 3.63 0.41 0.14 0.18 0.71 
PBT M2 CR 61 23 2.13 1.13 -0.5 4.87 0.61 0.26 0.22 1.15 
PBT M2 | 48 34 1.65 1.2 -0.17 4.87 0.5 0.25 0.18 1.15 
PBT M2 ll 14 4 2.14 0.18 1.94 2.34 0.63 0.09 0.51 0.7 
PBT M2 Ill 18 4 1.4 1.28 -0.5 2.27 0.6 0.14 0.47 0.79 
PBT M3 All 80 39 1.31 1.33 -2.27 4.61 0.56 0.26 0.13 1.24 
PBT M3 SSMC 12 12 0.73 1.67 -2.27 4.28 0.52 0.27 0.2 1.24 
PBT M3 CR 68 27 1.57 1.08 -0.35 4.61 0.58 0.26 0.13 1.1 
PBT M3 | 48 31 1.2 1.43 -2.27 4.61 0.54 0.27 0.13 1.24 
PBT M3 ll 14 4 1.62 0.48 0.96 1.99 0.72 0.14 0.52 0.83 
PBT M3 ll 18 4 1.87 0.95 0.71 2.99 0.53 0.23 0.25 0.76 


Note: M03 through M08 = mathematics grades 3 through 8, Al = Algebra |, GO = Geometry, A2 = Algebra II, M1 = Integrated Mathematics |, 
M2 = Integrated Mathematics II, M3 = Integrated Mathematics III. 
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Table A.12.1 Form Composition for ELA/L Grade 3 
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Claims Subclaims 

Reading 
Reading Literary Text 
Reading Informational Text 


Vocabulary 
Claim Total 

Writing 
Written Expression 
Knowledge of Conventions 
Claim Total 

SUMMATIVE TOTAL 


Note: This table is identical to Table 12.1 in Section 12. 


Table A.12.2 Form Composition for ELA/L Grade 4 


Number of Items 


10-12 
8-9 
5-8 

26 


29 


Number of Points 


23-27 

19-21 

10-16 
58 


Claims Subclaims 

Reading 
Reading Literary Text 
Reading Informational Text 
Vocabulary 
Claim Total 

Writing 
Written Expression 
Knowledge of Conventions 
Claim Total 

SUMMATIVE TOTAL 


Table A.12.3 Form Composition for ELA/L Grade 5 


Number of Items 


10-11 
10-12 
5-8 
28 


31 


Number of Points 


24 - 26 

24 - 28 

10-16 
64 


Claims Subclaims 

Reading 
Reading Literary Text 
Reading Informational Text 
Vocabulary 
Claim Total 

Writing 
Written Expression 
Knowledge of Conventions 
Claim Total 

SUMMATIVE TOTAL 
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Number of Items 


9-11 
9-11 
7-8 
28 


31 


Number of Points 


22 - 26 

22 - 26 

14-16 
64 
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Table A.12.4 Form Composition for ELA/L Grade 6 


Claims 
Reading 


Writing 


SUMMATIVE TOTAL 


2017 Technical Report 


Subclaims 


Reading Literary Text 
Reading Informational Text 
Vocabulary 

Claim Total 


Written Expression 
Knowledge of Conventions 
Claim Total 


Table A.12.5 Form Composition for ELA/L Grade 7 


Claims 
Reading 


Writing 


SUMMATIVE TOTAL 


Number of Items 


9-11 
14-16 
7-9 
34 


37 


Number of Points 


22 - 26 

32 - 36 

14-18 
76 


Subclaims 


Reading Literary Text 
Reading Informational Text 
Vocabulary 

Claim Total 


Written Expression 
Knowledge of Conventions 
Claim Total 


Table A.12.6 Form Composition for ELA/L Grade 8 


Claims 
Reading 


Writing 


SUMMATIVE TOTAL 
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Number of Items 


7-13 
13 - 16 
8-11 
34 


37 


Number of Points 


18 - 30 

32 - 36 

16-22 
76 


Subclaims 


Reading Literary Text 
Reading Informational Text 
Vocabulary 

Claim Total 


Written Expression 
Knowledge of Conventions 
Claim Total 


Number of Items 


7240 
19-49 
7=10 
34 


37 


Number of Points 


18 - 28 

28 - 42 

14-20 
76 
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Table A.12.7 Form Composition for ELA/L Grade 9 


Claims 
Reading 


Writing 


SUMMATIVE TOTAL 


2017 Technical Report 


Subclaims 


Reading Literary Text 
Reading Informational Text 
Vocabulary 

Claim Total 


Written Expression 
Knowledge of Conventions 
Claim Total 


Table A.12.8 Form Composition for ELA/L Grade 10 


Claims 
Reading 


Writing 


SUMMATIVE TOTAL 


Number of Items 


8-15 
12-18 
7=9 
34 


37 


Number of Points 


20 - 34 

28 - 40 

14-18 
76 


Subclaims 


Reading Literary Text 
Reading Informational Text 
Vocabulary 

Claim Total 


Written Expression 
Knowledge of Conventions 


Claim Total 


Table A.12.9 Form Composition for ELA/L Grade 11 


Claims 
Reading 


Writing 


SUMMATIVE TOTAL 
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Number of Items 


8-14 
13-19 
7-9 
34 


37 


Number of Points 


20 - 32 

30-42 

14-18 
76 


Subclaims 


Reading Literary Text 
Reading Informational Text 
Vocabulary 

Claim Total 


Written Expression 
Knowledge of Conventions 
Claim Total 


Number of Items 


8-14 
13:-19 
725 
34 


37 


Number of Points 


20 - 32 

30 - 42 

14-18 
76 
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Table A.12.10 Form Composition for Mathematics Grade 3 
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Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 

Note: This table is identical to Table 12.3 in Section 12. 


Table A.12.11 Form Composition for Mathematics Grade 4 

Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.12 Form Composition for Mathematics Grade 5 


Number of Items 


26 
10 


43 


Number of Items 


25 
8 


40 


Number of Points 


30 
10 
14 
12 
66 


Number of Points 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.13 Form Composition for Mathematics Grade 6 


Number of Items 


25 
8 


40 


Number of Points 


30 
10 
14 
12 
66 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 
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Number of Items 


20 
11 


38 


Number of Points 


26 
14 
14 
12 
66 
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Table A.12.14 Form Composition for Mathematics Grade 7 
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Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.15 Form Composition for Mathematics Grade 8 


Number of Items Number of Points 


23 29 
8 11 
14 

12 

38 66 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.16 Form Composition for Algebra | 


Number of Items Number of Points 


21 27 
8 13 
14 
12 
36 66 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.17 Form Composition for Geometry 


Number of Items Number of Points 


21 28 
13 21 
4 14 
4 18 
42 81 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 
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Number of Items Number of Points 


24 30 
14 19 
4 14 
4 18 
43 81 
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Table A.12.18 Form Composition for Algebra II 
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Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.19 Form Composition for Integrated Mathematics | 


Number of Items Number of Points 


20 29 
13 20 
4 14 
4 18 
41 81 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.20 Form Composition for Integrated Mathematics II 


Number of Items Number of Points 


21 31 
13 18 
4 14 
4 18 
42 81 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


Table A.12.21 Form Composition for Integrated Mathematics III 


Number of Items Number of Points 


22 32 
12 17 
4 14 
4 18 
42 81 


Subclaims 

Mathematics 
Major Content 
Additional & Supporting Content 
Expressing Mathematical Reasoning 
Modeling and Applications 

TOTAL 


March 27, 2018 


Number of Items Number of Points 


19 26 
13 23 
4 14 
4 18 
40 81 
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Appendix 12.2: Scaling Constants and Associated Information 


Table A.12.22 Threshold Scores and Scaling Constants for ELA/L Grades 3 to 8 


PARCC Assessment Threshold Cut Theta Scale Score A B 

Level 2 Cut -0.9648 700 
Level 3 Cut -0.2840 726 

Grade 3 ELA a  __ 36.7227 735.4297 
Level 4 Cut 0.3968 750 
Level 5 Cut 2.0360 810 
Level 2 Cut -1.3004 700 
Level 3 Cut -0.5079 725 

Grade 4 ELA ——|q.__ _ i —4J\A>\)\}m 31.5462 741.0214 
Level 4 Cut 0.2846 750 
Level 5 Cut 1.5578 790 
Level 2 Cut -1.3411 700 
Level 3 Cut -0.4924 726 

Grade 5 ELA 29.4580 739.5050 
Level 4 Cut 0.3563 750 
Level 5 Cut 2.0224 799 
Level 2 Cut -1.3656 700 
Level 3 Cut -0.4827 725 

Grade 6 ELA 28.3160 738.6673 
Level 4 Cut 0.4002 750 
Level 5 Cut 1.8133 790 
Level 2 Cut -1.2488 700 
Level 3 Cut -0.5117 725 

Grade 7 ELA SS 8.9161 742.3542 
Level 4 Cut 0.2254 750 
Level 5 Cut 1.2614 785 
Level 2 Cut -1.2730 700 
Level 3 Cut -0.5402 725 

Grade 8 ELA ——.|oaA umm  — A\+O4OOw—~=io i341183 743.4330 
Level 4 Cut 0.1925 750 
Level 5 Cut 1.4696 794 
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Table A.12.23 Threshold Scores and Scaling Constants for Mathematics Grades 3 to 8 


PARCC Assessment Threshold Cut Theta Scale Score A B 

Level 2 Cut -1.4141 700 
Level 3 Cut -0.6356 727 

Grade 3 Mathematics. _—@-——$£€ 32.1135 745.4119 
Level 4 Cut 0.1429 750 
Level 5 Cut 1.3931 790 
Level 2 Cut -1.3840 700 
Level 3 Cut -0.5484 727 

Grade 4 Mathematics. ———— 29.9167 741.4049 
Level 4 Cut 0.2873 750 
Level 5 Cut 1.8323 796 
Level 2 Cut -1.4571 700 
Level 3 Cut -0.5959 725 

Grade 5 Mathematics. ———— 29.0301 742.2997 
Level 4 Cut 0.2653 750 
Level 5 Cut 1.6262 790 
Level 2 Cut -1.3829 700 
Level 3 Cut -0.4948 725 

Grade 6 Mathematics. _—_—_——_ 28.1465 738.9252 
Level 4 Cut 0.3935 750 
Level 5 Cut 1.7567 788 
Level 2 Cut -1.4464 700 
Level 3 Cut -0.4505 725 

Grade 7 Mathematics ——— 25.1033 736.3102 
Level 4 Cut 0.5453 750 
Level 5 Cut 1.9919 786 
Level 2 Cut -0.8851 700 
Level 3 Cut -0.1264 728 

Grade 8 Mathematics. ——— _ _ _—__ 32.9505 729.1640 
Level 4 Cut 0.6323 750 
Level 5 Cut 2.1896 801 
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Table A.12.24 Threshold Scores and Scaling Constants for High School ELA 


POREE Threshold Cut Theta Scale Score A B 
Assessment 
Level 2 Cut -1.1635 700 
Level 3 Cut -0.4329 726 
Grade 9 ELA —— 34.2174 739.8124 
Level 4 Cut 0.2977 750 
Level 5 Cut 1.5065 791 
Level 2 Cut -0.8909 700 
Level 3 Cut -0.3112 725 
Grade 10ELA 43.1280 738.4223 
Level 4 Cut 0.2684 750 
Level 5 Cut 1.2858 794 
Level 2 Cut -1.1017 700 
Level 3 Cut -0.3859 726 
Grade 11 ELA.—_§_ —H—___ 34.9278 738.4801 
Level 4 Cut 0.3298 750 
Level 5 Cut 1.5206 792 
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Table A.12.25 Threshold Scores and Scaling Constants for High School Mathematics 


Eanes Threshold Cut Theta Scale Score A B 
Assessment 

Level 2 Cut -1.1781 700 
Level 3 Cut -0.3853 728 

Algebra | — a eee 31.5325 737.1490 
Level 4 Cut 0.4075 750 
Level 5 Cut 2.1651 805 
Level 2 Cut -0.5759 700 
Level 3 Cut 0.0860 726 

Algebra =-— a 37.7676 721.7509 
Level 4 Cut 0.7480 750 
Level 5 Cut 2.2728 808 
Level 2 Cut -1.3013 700 
Level 3 Cut -0.3389 726 

Geometry SS DEE SSDS 25.9775 733.8039 
Level 4 Cut 0.6235 750 
Level 5 Cut 1.8940 783 
Level 2 Cut -1.0919 700 
Level 3 Cut -0.3107 726 

Integrated 32.0043 734.9446 
Mathematics | Level 4 Cut 0.4704 750 
Level 5 Cut 1.9934 799 
Level 2 Cut -0.9175 700 
Level 3 Cut -0.0638 725 

Integrated 29.2865 726.8695 
Mathematics II Level 4 Cut 0.7898 750 
Level 5 Cut 1.9817 785 
Level 2 Cut -0.7076 700 
Level 3 Cut -0.0384 726 

Integrated 37.3549 726.4336 
Mathematics III Level 4 Cut 0.6309 750 
Level 5 Cut 2.0689 804 
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Table A.12.26 Scaling Constants for Reading and Writing Grades 3 to 11 


Reading Writing 
Ar Br Aw Bw 
Grade 3 ELA 14.6891 44.1719 7.3445 32.0859 
Grade 4 ELA 12.6184 46.4086 6.3093 33.2043 
Grade 5 ELA 11.7832 45.8019 5.8916 32.9010 
Grade 6 ELA 11.3264 45.4669 5.6632 32.7335 
Grade 7 ELA 13.5664 46.9416 6.7832 33.4708 
Grade 8 ELA 13.6472 47.3732 6.8237 33.6866 
Grade 9 ELA 13.6870 45.9250 6.8435 32.9625 
Grade 10 ELA 17.2512 45.3690 8.6256 32.6845 
Grade 11 ELA 13.9712 45.3920 6.9856 32.6961 
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Appendix 12.3: IRT Test Characteristic Curves, Information Curves, and CSEM Curves 


Figure A.12.1 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 3 
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Figure A.12.2 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 4 
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Figure A.12.3 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 5 
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Figure A.12.4 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 6 
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Figure A.12.5 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 7 
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Figure A.12.6 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 8 
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Figure A.12.7 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 9 
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Figure A.12.8 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 10 
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Figure A.12.9 IRT Test Characteristic Curves, Information Curves, and CSEM Curves ELA/L Grade 11 
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Figure A.12.10 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Mathematics Grade 3 
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Figure A.12.11 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Mathematics Grade 4 
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Figure A.12.12 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Mathematics Grade 5 
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Figure A.12.13 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Mathematics Grade 6 
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Figure A.12.14 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Mathematics Grade 7 
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Figure A.12.15 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Mathematics Grade 8 
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Figure A.12.16 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Algebra | 
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Figure A.12.17 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Geometry 
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Figure A.12.18 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Algebra II 
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Figure A.12.19 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Integrated Mathematics | 
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Figure A.12.20 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Integrated Mathematics II 
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Figure A.12.21 IRT Test Characteristic Curves, Information Curves, and CSEM Curves Integrated Mathematics III 
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Appendix 12.4: Subgroup Scale Score Performance 


Table A.12.27 Subgroup Performance for ELA/L Scale Scores: Grade 3 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 


N 
420,496 
206,149 
214,347 

5,920 
24,650 
73,056 

118,820 
672 
15,036 
181,382 
209,280 


210,597 


61,177 


349,974 
61,999 
357,072 
420,496 
206,149 
214,347 
5,920 
24,650 
73,056 
118,820 
672 
15,036 
181,382 
209,280 


210,597 


61,177 


349,974 
61,999 
357,072 
420,496 
206,149 
214,347 


Mean 


738.23 
743.10 


733.54 
716.34 
766.38 
721.93 
725.02 
745.66 
743.81 
749.85 
722.50 


753.81 


712.91 


742.44 

709.57 

743.21 
45.36 
46.67 
44.10 
36.35 
55.60 
38.82 
39.74 
47.29 
47.96 
50.37 
38.98 


51.69 


34.91 


47.09 
34.76 
47.21 
30.41 
32.48 
28.43 


2017 Technical Report 


SD 


41.04 
41.14 


40.40 
34.04 
39.94 
38.49 
37.91 
42.15 
40.77 
38.43 
37.11 


38.74 


34.21 


40.55 
38.69 
39.37 
16.33 
16.26 
16.31 
13.27 
15.92 
14.94 
14.69 
16.35 
16.36 
15.55 
14.45 


15.62 


13.02 


16.17 
15.44 
15.77 
12.53 
12.13 
12.58 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 


Max 


850 
850 


850 
850 
850 
850 
850 
850 
850 
850 
850 


850 


850 


850 

850 

850 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 


90 


90 


90 
90 
90 
60 
60 
60 
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Group Type Group N Mean SD Min Max 
“Ethnicity -—~—=—=—=SAmericanIndian/Alaska Native «5,920. 25.71 «1167, 10's 
‘Asan t—“‘ éSO™;™*;*;”;”!”!”#*#DMOSOO!#OC3B19°©«10940=6«6100~*é‘iO 
“Black or African American —=—=~*«~«S«05H.—«C*«iGSOCBSC*«‘dCSC“‘<‘éC SCS” 
“Hispanic/Latino ~—«*'118,820. «27.68 +=91238~=6102Sti«iO 
Native Hawaiian or Pacific Islander 672 33.23 12.62 10 60 
“Multiple Race Selected +~«15,036+=«31.22««1246~=St«C«dsti<‘<«é‘i CS 
“Whie 181,382 3280 1182 10 60 | 
Economic Status” Economically Disadvantaged 209,280 26.71 = 12.29 10 60 
Not Economically Disadvantaged 210,597, 34.09 11.64 10 60 
a ld English Learner (EL) 61,177. 24.79 12.01 10 60 
Non English Learner 349,974 31.36 12.36 10 60 
“Disabilities +~—-«Students with Disabilities (SWD) «61,999 2199 1227, 10s GO 
“Students without Disabilities ..+«357,072-«31.88 «1198 + #10 60 


Note: This table is identical to Table 12.7 in Section 12. 
“Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or reduced- 
price lunch (FRL). 
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Table A.12.28 Subgroup Performance for ELA/L Scale Scores: Grade 4 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
423,098 
207,393 
215,705 
5,712 
24,252 
71,790 
121,196 
712 
14,484 
183,920 
208,196 


214,097 


39,762 


373,645 
66,696 
354,908 
423,098 
207,393 
215,705 
5,712 
24,252 
71,790 
121,196 
712 
14,484 
183,920 
208,196 


214,097 
39,762 


373,645 
66,696 
354,908 
423,098 
207,393 
215,705 
5,712 
24,252 


Mean 
741.99 
746.75 
737.41 
719.88 
767.86 
727.19 
730.79 
752.00 
746.25 
752.05 
727.96 


755.60 


709.56 


745.22 
714.35 
747.19 
46.67 
47.83 
45.55 
37.56 
56.39 
40.76 
41.,89 
49.75 
48.71 
50.95 
40.99 


52.17 
3364 


47.96 
36.62 
48.55 
32.27 
34.17 
30.44 
27.45 
38.70 


2017 Technical Report 


SD 
35.79 
35.54 
35.43 
30.85 
34.55 
33.47 
32.91 
34.01 
35.67 
33.49 
32.43 


33.56 


27.58 


34.80 
34.61 
33.55 
14.22 
14.08 
14.25 
12.20 
13.81 
12.93 
12.79 
13.31 
14.24 
13.49 
12.65 


13.47 
10.45 


13.85 
13.77 
13.49 
10.26 
9.67 
10.47 
9.85 
8.85 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


10 
10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 


850 


849 


850 

850 

850 
90 
90 
90 
88 
90 
90 
90 
90 
90 
90 
90 


90 
90 


90 
90 
90 
60 
60 
60 
60 
60 
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Group Type Group N Mean SD Min Max 
Black or African American 71,790 28.74 10.46 10 60 
“Hispanic/Latino ~+«-121,196 +=«-29.94 1018 = 10'siC«iO 
Native Hawaiian or Pacific Islander 712 35.50 9.20 10 60 
“Multiple RaceSelected +—«:14,484+~=«32.90 1019 «102s 
“Whie 183920 3442 946 10 60 | 
Economic Status” Economically Disadvantaged 208,196 29.03 10.19 10 60 
Not Economically Disadvantaged 214,097 35.41 9.30 10 60 
ee ees English Learner (EL) 39,762 24.53 9.97 10 60 
Non English Learner 373,645 33.04 9.94 10 60 
“Disabilities +~—-Students with Disabilities (SWD) 66,696 2430 1096 10 60 | 
“Students without Disabilities +«-354,908 33.77. «939 ~«10St*té«<‘ CO 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 


March 27, 2018 Page 368 


PARCC 


Table A.12.29 Subgroup Performance for ELA/L Scale Scores: Grade 5 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
414,461 
203,177 
211,284 
5,579 
24,081 
68,895 
117,135 
640 
13,586 
183,710 
199,992 


213,846 


29,084 


375,848 
67,384 
345,765 
414,461 
203,177 
211,284 
5,579 
24,081 
68,895 
117,135 
640 
13,586 
183,710 
199,992 


213,846 


29,084 


375,848 
67,384 
345,765 
414,461 
203,177 
211,284 
5,579 
24,081 


Mean 
742.72 
748.37 
737.28 
723.58 
767.80 
727.80 
731.87 
749.92 
747.21 
752.17 
728.95 


755.57 


706.57 


745.29 

715.48 

748.03 
47.01 
48.41 
45.65 
39.38 
56.23 
41.27 
42.39 
48.80 
49.30 
50.95 
41.53 


52.12 


32.59 


48.04 
36.94 
48.97 
31.35 
33.85 
28.95 
26.65 
38.44 


2017 Technical Report 


SD 
34.77 
34.54 
34.11 
29.76 
33.97 
32.54 
32.37 
36.22 
34.80 
32.27 
31.70 


32.51 


26.54 


33.75 
32.54 
32.65 
13.93 
13.80 
13.91 
11.89 
13.66 
12.89 
12.84 
13.94 
14.04 
13.05 
12.60 


13.13 


10.27 


13.55 
13.13 
13.21 
11.45 
10.60 
11.73 
10.84 
9.54 


Min 
650 
650 
650 
650 
650 
650 
650 
654 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
832 
850 
847 
850 
837 
850 
850 
849 


850 


827 


850 

850 

850 
90 
90 
90 
87 
90 
90 
90 
81 
90 
90 
90 


90 


81 


90 
90 
90 
60 
60 
60 
60 
60 
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“GroupType Group  .§= § +N. Mean SD. Min Max | 
Black or African American =—(“aitst(iséi BSS 269GH.——s1' BCC 
“Hispanic/Latino ~—«:117,135.=«28.83.«1142,—s10'sti‘i 
Native Hawaiian or Pacific Islander 640 34.00 11.58 10 57 
“Multiple RaceSelected —« 13,586 31.99 1139S 10''si« 
“Whie 183,710 33.76 1050 10 60 | 
Economic Status” Economically Disadvantaged 199,992 27.63 11.51 10 60 
Not Economically Disadvantaged 213,846 34.83 10.24 10 60 
say al English Learner (EL) 29,084 2137 1092 10 57 
Non English Learner 375,848 32.06 11.14 10 60 
“Disabilities +—-«Students with Disabilities (SWD) -—«67,384«S «22.46 «1188S 10s 
“Students without Disabilities +«-345,765 «33.09 «1053.+2=«10s'sti‘<‘ ; 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 
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Table A.12.30 Subgroup Performance for ELA/L Scale Scores: Grade 6 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
406,705 
198,893 
207,812 

5,044 
24,173 
66,.708 
114,236 

664 
12,609 
182,624 
191,905 


214,145 


24,074 


373,441 
66,115 
338,883 
406,705 
198,893 
207,812 
5,044 
24,173 
66,708 
114,236 
664 
12,609 
182,624 
191,905 


214,145 


24,074 


373,441 
66,115 
338,883 
406,705 
198,893 
207,812 
5,044 
24,173 


Mean 
741.20 
747.48 
735.19 
725.25 
765.99 
726.83 
731.27 
749.83 
744.11 
749.62 
728.40 


752.68 


707.42 


743.30 

715.94 

746.17 
46.40 
48.04 
44.82 
39.44 
55.53 
40.96 
42.26 
48.88 
47.93 
49.85 
41.33 


50.93 


33.17 


47.21 
37.20 
48.20 
30.97 
33.90 
28.17 
27.72 
38.19 


2017 Technical Report 


SD 
31.87 
31.42 
31.13 
27.12 
31.58 
28.74 
28.85 
32.01 
32.12 
30.19 
28.30 


30.48 


21.87 


31.19 
28.59 
30.09 
12.44 
12.32 
12.34 
10.35 
12.36 
11.03 
11.05 
12.21 
12.67 
11.93 
10.87 


12.00 


7.92 


12.20 
11.20 
11.86 
11.26 
10.08 
11.62 
10.84 
9.41 


Min 
650 
650 
650 
650 
650 
650 
650 
664 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
17 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
830 
850 
850 
850 
831 
850 
850 
850 


850 


821 


850 

850 

850 
90 
90 
90 
80 
90 
90 
90 
79 
90 
90 
90 


90 


78 


90 
90 
90 
60 
60 
60 
60 
60 
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“GroupType Group = = N Mean SD. Min Max | 

Black or African American =(asi(téi Bes s1.46— 10 (tO 
Hispanic/Latino 114,236 28.51 11.21 10 60 
Native Hawaiian or Pacific Islander 664 34.24 10.80 10 60 

“Multiple RaceSelected ~—=«-12,609.-=31.30.1128+=S10''si« 

“White 182,624 33.23 1039 10 £60 
Economic Status” Economically Disadvantaged 191,905 27.39 = 11.29 10 60 
Not Economically Disadvantaged 214,145 = 34.20 10.20 10 60 
say cal English Learner (EL) 24,074 2080 1079 10 60 
Non English Learner 373,441 31.62 10.97 10 60 

“Disabilities +—-Students with Disabilities (SWD) «66,115 «22.21 «1160, 10s 
Students without Disabilities 338,883 32.70 10.35 10 60 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 
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Table A.12.31 Subgroup Performance for ELA/L Scale Scores: Grade 7 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
406,201 
198,297 
207,904 

4,827 
24,425 
66,386 

111,338 
561 
12,042 
186,004 
185,213 


220,327 


23,215 


374,221 
65,523 
339,130 
406,201 
198,297 
207,904 
4,827 
24,425 
66,386 
111,338 
561 
12,042 
186,004 
185,213 


220,327 


23,215 


374,221 
65,523 
339,130 
406,201 
198,297 
207,904 
4,827 
24,425 


Mean 
743.24 
751.57 
735.31 
721.80 
772.55 
726.68 
730.89 
753.27 
745.80 
753.10 
727.72 


756.31 


700.53 


745.80 

712.24 

749.29 
47.02 
49.25 
44.89 
38.32 
57.69 
40.71 
41.92 
49.89 
48.62 
51.05 
40.92 


52.15 


30.12 


48.03 
35.64 
49.23 
32.22 
35.42 
29.17 
27.30 
40.13 


2017 Technical Report 


SD 
38.24 
37.05 
37.67 
33.04 
37.08 
35.22 
35.62 
39.52 
37.69 
35.62 
34.82 


36.04 


27.75 


37.21 
34.94 
35.87 
15.06 
14.66 
15.12 
12.83 
14.85 
13.67 
13.84 
15.00 
15.04 
14.18 
13.54 


14.34 


10.23 


14.68 
13.87 
14.26 
11.68 
10.50 
11.92 
11.13 
9.87 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 


850 


850 


850 

850 

850 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 


90 


90 


90 
90 
90 
60 
60 
60 
60 
60 
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English L 23,215 21.07 10.86 10 60 
Scab English Learner (EL) ’ 
Status 
Non English Learner 374,221 32.89 11.37 10 60 
Disabilities Students with Disabilities (SWD) 65,523 22.92 = 11.88 10 60 
Students without Disabilities 339,130 34.03 10.74 10 60 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 
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Table A.12.32 Subgroup Performance for ELA/L Scale Scores: Grade 8 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
399,371 
194,852 
204,519 

4,579 
23,961 
65,952 
108,721 

602 

11,226 
183,788 
179,179 


219,525 


22,686 


368,666 
63,687 
334,211 
399,371 
194,852 
204,519 
4,579 
23,961 
65,952 
108,721 
602 
11,226 
183,788 
179,179 


219,525 


22,686 


368,666 
63,687 
334,211 
399,371 
194,852 
204,519 
4,579 
23,961 


Mean 
742.13 
750.99 
733.68 
723.18 
773.28 
725.07 
730.67 
752.76 
744.91 
751.26 
726.86 


754.62 


700.92 


744.59 

711.06 

748.10 
46.94 
49.42 
44.58 
38.71 
58.62 
40.49 
42.05 
50.07 
48.78 
50.72 
40.84 


51.93 


30.34 


47.93 
35.24 
49.19 
31.97 
35.19 
28.90 
28.25 
40.01 


2017 Technical Report 


SD 
39.11 
37.83 
38.44 
32.78 
38.19 
35.72 
35.77 
38.69 
39.58 
37.22 
35.20 


37.68 


27.37 


38.29 
35.29 
36.94 
15.61 
15.24 
15.60 
12.97 
15.47 
14.11 
14.10 
15.17 
16.04 
15.00 
13.88 


15.17 


10.15 


15.31 
14.09 
14.88 
11.51 
10.36 
11.71 
10.49 
9.94 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 


850 


850 


850 

850 

850 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 


90 


90 


90 
90 
90 
60 
60 
60 
60 
60 
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“GroupType + +Group i (ststi“‘<‘<‘;é;SO!O!O!U(UdNC~CO*«xMecan~—=6 SD) OMin~— Max 
Black or African American =—(asi(ié‘i SSA 1B 1 
“Hispanic/Latino ~—«*'108,721.+=«Co29.46—Ss1.22,2S—«<Y200s'i‘<iésOS 
Native Hawaiian or Pacific Islander 602 35.49 10.89 10 60 
“MultipleRaceSelected ~—=«.11,226~=«32.04Ss1151 Ss 10'ié«O 
“White 183,788 34.15 1083 10 60 
Economic Status” Economically Disadvantaged 179,179 28.17, 11.30 10 60 
Not Economically Disadvantaged 219,525 35.08 10.72 10 60 
say al English Learner (EL) 22,686 2154 1048 10 60 
Non English Learner 368,666 32.60 11.26 10 60 
“Disabilities +~—-«Students with Disabilities (SWD) -—«63,687=«s 23.15.1160 10s GO 
“Students without Disabilities. +«-334,211 «33.67. +«1070~2=C*«dsti(i«i 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 
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Table A.12.33 Subgroup Performance for ELA/L Scale Scores: Grade 9 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
188,201 
90,951 
97,250 
3,407 
12,192 
21,038 
62,335 
373 
3,839 
84,482 
78,341 


109,453 


15,584 


166,483 
30,113 
157,483 
188,201 
90,951 
97,250 
3,407 
12,192 
21,038 
62,335 
373 
3,839 
84,482 
78,341 


109,453 


15,584 


166,483 
30,113 
157,483 
188,201 
90,951 
97,250 
3,407 
12,192 


Mean 
740.92 
748.89 
733.47 
719.95 
771.45 
727.01 
726.81 
749.01 
744.07 
751.22 
725.35 


752.17 


698.36 


745.00 

713.88 

746.20 
46.54 
48.72 
44.50 
37.37 
57.70 
40.82 
40.66 
49.35 
48.46 
51.02 
40.14 


51.15 


29.33 


48.14 
36.22 
48.55 
30.89 
34.13 
27.86 
26.58 
39.46 


2017 Technical Report 


SD 
37.97 
36.81 
37.53 
30.27 
37.59 
34.98 
35.14 
37.05 
37.68 
34.90 
34.39 


36.42 


26.51 


36.44 
34.03 
36.43 
15.34 
14.97 
15.40 
12.13 
15.49 
13.91 
13.92 
14.90 
15.45 
14.25 
13.66 


14.81 


10.03 


14.75 
13.79 
14.81 
11.86 
10.64 
12.14 
10.85 
9.95 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
828 
850 
850 
850 
850 
850 
850 
850 


850 


846 


850 

850 

850 
90 
90 
90 
79 
90 
90 
90 
90 
90 
90 
90 


90 


90 


90 
90 
90 
60 
60 
60 
56 
60 
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“GroupType + +Groupsti(<‘é‘éséé!O!O!!!U(UN.~O€C€UM)lrn~ CSD CUM Max 
Black or African American =——(is=<i«éBBesCASs1.760sd10—( ss 
“Hispanic/latino.=—=—s—=“‘é«*«*«i;SS:*«*UT ASSOC COCO 
Native Hawaiian or Pacific Islander 363 33.33 11.47 10 60 
“MultipleRaceSelected 3,839 34.02 1184 10 £60 
“White 84482 33.28 1092 10 60 
Economic Status” Economically Disadvantaged 78,341 26.95 11.76 10 60 
Not Economically Disadvantaged 109,453 33.76 11.07 10 60 
ay cal English Learner (EL) 15584 19.49 1069 10 60 
Non English Learner 166,483 32.04 11.37 10 60 
“Disabilities | +—~—»-Students with Disabilities (SWD) «30,113 22.92 11.79 10s 
“Students without Disabilities .+«157,483.~«32.45.°«1123.°2=«10ststi‘<‘C; CO” 


Note: This table is identical to Table 12.8 in Section 12. 
“Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or reduced- 
price lunch (FRL). 
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Table A.12.34 Subgroup Performance for ELA/L Scale Scores: Grade 10 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
171,965 
83,918 
88,047 
2,750 
13,131 
36,910 
46,128 
247 
3,625 
68,720 
67,073 


104,321 


9,521 


161,970 
28,567 
143,757 
171,965 
83,918 
88,047 
2,750 
13,131 
36,910 
46,128 
247 
3,625 
68,720 
67,073 


104,321 


9,521 


161,970 
28,567 
142,757 
171,965 
83,918 
88,047 
2,750 
13,131 


Mean 
741.35 
750.44 
732.70 
721.90 
775.59 
724.52 
725.90 
753.37 
753.99 
754.51 
723.12 


753.25 


684.99 


744.75 

709.11 

747.94 
46.84 
49.30 
44.50 
37.67 
59.53 
40.56 
40.49 
50.52 
52.43 
52.20 
39.53 


51.61 


24.89 


48.16 
34.78 
49.31 
31.48 
34.70 
28.41 
28.49 
40.26 


2017 Technical Report 


SD 
46.94 
45.41 
46.74 
37.66 
44.18 
43.33 
43.77 
46.78 
45.85 
43.81 
42.85 


45.65 


29.42 


45.65 
41.95 
45.16 
18.67 
18.14 
18.87 
15.22 
17.95 
17.22 
17.16 
18.15 
18.57 
17.59 
16.81 


18.27 


11.24 


18.21 
16.89 
18.05 
13.05 
12.18 
13.12 
11.11 
11.43 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 


850 


850 


850 

850 

850 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 
90 


90 


90 


90 
90 
90 
60 
60 
60 
60 
60 
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“GroupType Group .§= = +N. Mean SD Min Max | 
Black or African American =“ (issi(itéi 2 9ss27.09s12'SBsCsC( tC 
“Hispanic/latino.=—=—Ss=<“<i«‘;*«S SS AZB;*O*«iO:S*«‘iaC:SC“‘d‘SC“‘<‘éiRSCS*” 
Native Hawaiian or Pacific Islander 247 35.29 12.62 10 60 
“MultipleRaceSelected 3,625 3419 1241 #2410 60 
“Whie 68720 3451 1225 10 60 | 
Economic Status” Economically Disadvantaged 67,073 27.18 = 12.60 10 60 
Not Economically Disadvantaged 104,321 34.30 12.56 10 60 
say cal English Learner (EL) 9,521 1737 982 10 60 
Non English Learner 161,970 32.33 12.74 10 60 
“Disabilities +~—-«Students with Disabilities (SWD) «28,567. 22.91 1229 «10s 
“Students without Disabilities +«142,757.—«33.23~=«12:50.S«10s'ti«<C 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 
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Table A.12.35 Subgroup Performance for ELA/L Scale Scores: Grade 11 


Group Type 


Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Reading Score 


Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 


Gender 


Ethnicity 


March 27, 2018 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


Female 
Male 
American Indian/Alaska Native 


Asian 


N 
107,593 
51,199 
56,394 
2,961 
6,512 
21,367 
34,428 
168 
1,491 
40,252 
45,210 


62,173 


5,288 


102,058 
19,094 
88,265 

107,593 
51,199 
56,394 

2,961 
6,512 
21,367 
34,428 
168 
1,491 

40,252 

45,210 


62,173 


5,288 


102,058 
19,094 
88,265 

107,593 
51,199 
56,394 

2,961 
6,512 


Mean 
735.26 
743.05 
728.19 
732.22 
756.36 
723.67 
732.48 
745.58 
740.59 
740.54 
728.62 


740.13 


701.08 


737.06 

711.38 

740.46 
44.44 
46.52 
42.55 
41.20 
52.68 
39.94 
42.77 
47.56 
47.19 
47.12 
41.28 


46.75 


30.63 


45.16 
35.48 
46.39 
28.46 
31.74 
25.48 
30.39 
34.20 


2017 Technical Report 


SD 
39.46 
38.61 
38.89 
33.58 
41.69 
37.56 
37.52 
37.98 
39.82 
39.75 
37.54 


40.13 


27.83 


39.18 
34.59 
38.54 
15.36 
15.03 
15.42 
12.66 
16.70 
14.33 
14.33 
14.24 
15.91 
15.61 
14.21 


15.76 


9.94 


15.26 
13.48 
15.06 
13.41 
12.80 
13.25 
11.71 
13.12 


Min 
650 
650 
650 
650 
650 
650 
650 
656 
650 
650 
650 


650 


650 


650 

650 

650 
10 
10 
10 
10 
10 
10 
10 
14 
10 
10 
10 


10 


10 


10 
10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
850 
850 
850 
850 
838 
850 
850 
850 


850 


850 


850 

850 

850 
90 
90 
90 
90 
90 
90 
90 
84 
90 
90 
90 


90 


87 


90 
90 
90 
60 
60 
60 
60 
60 
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“GroupType Group = §= +N. Mean SD Min Max | 
Black or African American =——(asti(tséi BOTH 18.22 s0—( ti 
Hispanic/Latino 34,428 28.48 13.04 10 60 
Native Hawaiian or Pacific Islander 168 32.32 13.05 10 60 
“MultipleRaceSelected 1491 29.21 1329 #10 60 
“Whie 40,252 2910 1354 10 60 | 
Economic Status” Economically Disadvantaged 45,210 27.31 13.21 10 60 
Not Economically Disadvantaged 62,173 29.31 13.49 10 60 
say cal English Learner (EL) 5,288 1976 1129 10 60 
Non English Learner 102,058 28.92 13.36 10 60 
“Disabilities +—-Students with Disabilities (SWD) -—«:19,094 21.34 «1233.10 
“Students without Disabilities +«88,265+~«=«30.01.—=«s13.43':—«‘a‘ti it 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 
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Table A.12.36 Subgroup Performance for Mathematics Scale Scores: Grade 3 


Group Type Group N Mean SD Min Max 
“FullSummative Score =~ ~~——~<C~*~C‘“Cs~::::::S«S TAZ 37LOB.-—««GHOD.s—i«SO 
“Gender —sté<‘éiemmadilc”«*=™=™=™©™©™©™)™!™~™~”!”!~!~:«208;739°~«742.47 «35.83 6500S 850 

“Male =——t—<i‘isSOOOSOO”C“‘«i‘éCé‘éNT,OOS;:~=«O741«9G 3B.25°~=«—S5OSCOS 

“Ethnicity. ——=—=~=~S*CAmerican Indian/Alaska Native -—=s«5,920724.13.« 31.47 650.845 

‘Asian t—“‘“‘éSOSCS;*;*;*;*;”;”!”*#é#SY«ABQ*O77299 936.36 ©6502 850 

“Black or AfricanAmerican —=—=~*«i738«266H—=«726.20 34.03. 650-850 

“Hispanic/Latino ~—~—~—~—«*122,975.-« 730.60. «33.67 650. 850 
Native Hawaiian or Pacific Islander 684 746.92 38.61 650 850 

“Multiple RaceSelected —~—=«15,061.-—«-745.90 37.32 650 850 

“White 181,746 752.53 3461 650 850 
Economic Status” Economically Disadvantaged 213,460 728.31 33.58 650 850 
aa es English Learner (EL) 66,662 723.87 33.05 650 850 
Non English Learner 349,729 745.55 36.74 650 850 

“Disabilities | +~—-‘Students with Disabilities (SWD) «62,252. 718.87 37.60 650. 850_~ 
Students without Disabilities 362,041 746.23 35.47 650 850 


Note: This table is identical to Table 12.9 in Section 12. 
“Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or reduced- 
price lunch (FRL). 
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Table A.12.37 Subgroup Performance for Mathematics Scale Scores: Grade 4 


Group Type 
Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


N 
426,609 
209,075 
217,534 

5,705 
24,736 
71,810 

123,935 
715 
14,473 
184,163 
210,906 


214,884 


43,770 


373,143 
66,741 
358,337 


Mean 
737.10 
737.16 
737.05 
718.38 
766.79 
721.35 
726.31 
744.38 
740.33 
746.82 
723.79 


750.13 


710.80 


740.02 
715.00 
741.22 


2017 Technical Report 


SD 
33.55 
32.72 
34.34 
27.87 
33.47 
30.17 
29.81 
33.79 
33.66 
31.31 
29.72 


31.92 


26.21 


32.96 
32.25 
32.15 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 
650 
650 


Max 
850 
850 
850 
850 
850 
850 
850 
841 
850 
850 
850 


850 


850 


850 
850 
850 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 


reduced-price lunch (FRL). 
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Table A.12.38 Subgroup Performance for Mathematics Scale Scores: Grade 5 


Group Type 
Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


N 
416,556 
204,220 
212,336 

5,580 
24,451 
68,885 

118,549 
644 
13,600 
183,969 
201,378 


214,549 


31,670 


375.372 
67,302 
347,916 


Mean 
737.24 
737.79 
736.70 
720.52 
766.83 
721.33 
727.02 
743.80 
740.50 
746.12 
724.29 


749.37 


709.73 


739.41 
716.02 
741.36 


2017 Technical Report 


SD 
32.52 
31.17 
33.76 
26.88 
33.01 
28.61 
28.67 
33.68 
33.39 
30.54 
28.50 


31.34 


25.56 


31.96 
30.35 
31.30 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 
650 
650 


Max 
850 
850 
850 
845 
850 
850 
850 
850 
850 
850 
850 


850 


850 


850 
850 
850 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 


reduced-price lunch (FRL). 
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Table A.12.39 Subgroup Performance for Mathematics Scale Scores: Grade 6 


Group Type 
Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


N 
408,664 
199,820 
208,844 

5,042 
24.338 
66,837 

115,760 
662 
12,593 
182,724 
193,487 


214,515 


26,712 


372,753 
66,046 
340,885 


Mean 
734.27 
735.71 
732.90 
717.64 
763.71 
717.35 
723.53 
741.97 
737.29 
743.60 
720.71 


746.51 


703.04 


736.38 
711.27 
738.76 


2017 Technical Report 


SD 
32.60 
31.57 
33.49 
27.43 
33.14 
28.51 
28.72 
32.04 
33.35 
30.26 
28.45 


31.22 


25.01 


31.92 
29.84 
31.20 


Min 
650 
650 
650 
650 
650 
650 
650 
653 
650 
650 
650 


650 


650 


650 
650 
650 


Max 
850 
850 
850 
850 
850 
850 
850 
825 
850 
850 
850 


850 


850 


850 
850 
850 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 


reduced-price lunch (FRL). 
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Table A.12.40 Subgroup Performance for Mathematics Scale Scores: Grade 7 


Group Type 
Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


N 
393,942 
192,605 
201,337 

4,796 
21,135 
64,993 
111,808 
556 
11,406 
178,577 
184,824 


208,531 


25,770 


360,002 
64,851 
327,631 


Mean 
733.72 
734.89 
732.61 
717.58 
757.71 
719.87 
724.67 
740.00 
735.07 
741.95 
722.52 


743.66 


707.16 


735.56 
713.13 
737.83 


2017 Technical Report 


SD 
28.66 
27.81 
29.40 
24.23 
29.39 
25.22 
25.81 
31.37 
28.52 
26.88 
25.53 


27.60 


22.60 


28.11 
26.97 
27.20 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 
650 
650 


Max 
850 
850 
850 
815 
850 
850 
850 
818 
850 
850 
850 


850 


842 


850 
850 
850 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 


reduced-price lunch (FRL). 
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Table A.12.41 Subgroup Performance for Mathematics Scale Scores: Grade 8 


Group Type 
Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


N 
314,484 
151,133 
163,351 
4,122 
19711 
56,058 
94,511 
431 
8,480 
137,601 
159,178 


154,827 


23,917 


284,477 
58,599 
254,694 


Mean 
725.57 
728.13 
723.20 
711.03 
756.71 
708.85 
716.61 
732.96 
727.93 
735.96 
714.27 


737.23 


697.30 


727.85 
701.62 
731.13 


2017 Technical Report 


SD 
36.97 
36.05 
37.64 
30.63 
41.20 
31.90 
33.24 
37.09 
38.36 
35.93 
33.23 


37.01 


27.68 


36.65 
32.08 
35.81 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 
650 
650 


Max 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 
850 


850 


850 


850 
850 
850 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 


reduced-price lunch (FRL). 
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Table A.12.42 Subgroup Performance for Mathematics Scale Scores: Algebra | 


Group Type Group N Mean SD Min Max 
“Gender sté‘émdics”«*;*#*!~!~!~!~!~!~!”!”!~!~C(26;562 «739.22 «33.69 = 650850 
“Male =—t—<“<i—si‘:;~™*™*™*~*~*:”:”C*C«dS AS27387 3OT——«SO—Cié«iSO 
‘Asian t—=<“‘éSOC;*;*;*;”!”!”#*«O79°~«770.120©=«36.402~« G50—(ts«S 
“Hispanic/Latino + 74,965 724.45 29.81 650 850 
Native Hawaiian or Pacific Islander 452 746.57 = 35.29 650 850 
“White 111,446 748.89 3287 650 850 
“Not Economically Disadvantaged 153,948 748.42 34.85 650 850 — 
aa oe English Learner (EL) 20,325 707.78 24.46 650 850 
Non English Learner 235,967 740.64 34.45 650 850 
Students without Disabilities 219,187 742.10 34.37 650 850 


Note: This table is identical to Table 12.10 in Section 12. 
“Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or reduced- 
price lunch (FRL). 
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Table A.12.43 Subgroup Performance for Mathematics Scale Scores: Geometry 


Group Type 
Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


N 
137,530 
67,407 
70,123 
2,683 
11,018 
18,568 
40,276 
227 
2,537 
61,782 
50,295 


86,682 


7,054 


128,566 
20,741 
116,084 


Mean 
734.93 
735.42 
734.46 
721.24 
759.75 
719.22 
723.41 
741.45 
744.66 
743.06 
721.59 


742.76 


709.24 


736.13 
715.33 
738.54 


2017 Technical Report 


SD 
27.79 
26.83 
28.69 
21.31 
26.88 
23.89 
23.69 
24.88 
27.92 
25.39 
23.50 


27.13 


20.40 


27.42 
24.88 
26.80 


Min 
650 
650 
650 
650 
650 
650 
650 
671 
650 
650 
650 


650 


650 


650 
650 
650 


Max 
850 
850 
850 
803 
850 
837 
850 
825 
830 
850 
830 


850 


844 


850 
830 
850 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 


reduced-price lunch (FRL). 
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Table A.12.44 Subgroup Performance for Mathematics Scale Scores: Algebra II 


Group Type 
Full Summative Score 


Gender 


Ethnicity 


Economic Status* 


English Learner 
Status 


Disabilities 


Group 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


N 
122,710 
61,281 
61,429 
2,554 
10,596 
21,403 
33,483 
203 
2,245 
51,809 
43,516 


78,920 


4,544 


117,481 
15,431 
106,970 


Mean 
723.30 
723.18 
723.42 
703.78 
760.57 
702.08 
709.10 
732.89 
733.06 
734.35 
706.64 


732.49 


690.65 


724.41 
698.34 
726.91 


2017 Technical Report 


SD 
40.84 
39.05 
42.55 
29.02 
43.34 
32.00 
33.05 
40.78 
41.23 
39.42 
32.76 


41.94 


29.05 


40.65 
33.98 
40.49 


Min 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 
650 


650 


650 


650 
650 
650 


Max 
850 
850 
850 
829 
850 
850 
850 
825 
850 
850 
850 


850 


850 


850 
850 
850 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 


reduced-price lunch (FRL). 
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Table A.12.45 Subgroup Performance for Mathematics Scale Scores: Integrated Mathematics | 


“GroupType + +Group =  N. Mean SD Min. Max 
“FullSummativeScore  =~=——s=<CsSs‘i=‘C:*‘:;:;*«*NGA:S=«CZOAO=8B.TZ—s«SOSC«C«SO 
“Gender =—sté‘é‘smadil@”=O*™”!™!™!™!™!™!™!™!™!™!”!”!~!~”~(~S 09573086 «32.94 = 50—s«AC 
“Male =—t—<issti‘—s.UC(Csti‘<CS~*:*~<~S* BG 73.0 344BB-s«*wSSHD—“(ws«S 
“Ethnicity —~—=—=—=~=S=«CAmerican Indian/Alaska Native += 149.—=Ss 716.07 «26.71 = 650s 790~ 
‘Asian —t—“‘é‘OCOC*;*;*;*;*;”;”!”!”#@@3L~«é«=74S.92~~«<39.15~=~«=«*isS SCS 
“BlackorAfricanAmerican ~—~—~—=«809~S—«s72.63.-«30.23.650.—Ss«SS 
“Hispanic/Latino ~—~—~—«S,272—«720.80 29.02 650i 
Native Hawaiian or Pacific Islander 28 748.39 = 42.15 673 810 
“Multiple Race Selected ~—~—=«272~=«a7384«92_-—=«s3B.B7—s—«iSSC*«iS OS 
“White 3,997 743.29 3467 650 850 | 
Economic Status” Economically Disadvantaged 5,793 721.21 29.68 650 838 
Not Economically Disadvantaged 5,164 740.77 34.94 650 850 
aa es English Learner (EL) 2,191 707.58 21.98 650 815 
Non English Learner 6,506 734.82 32.28 650 850 
“Disabilities -—~—=«Students with Disabilities (SWD) +«',356-- 709.29. «28.29 += 650-830 
Students without Disabilities 9,601 733.42 33.34 650 848 


Note: This table is identical to Table 12.11 in Section 12. 
“Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or reduced- 
price lunch (FRL). 
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Table A.12.46 Subgroup Performance for Mathematics Scale Scores: Integrated Mathematics II 


Group Type Group N Mean SD Min Max 
“Fullsummative Score ~~—~—~—~—~—~«R«*74S.AG 32.67 «650-50 
“Gender =—sté<‘éiomaile@:”«=*;*™!™©™©™©™©™™U™!™!™!~!~!”~!~C(«diNO2:~«O745.34 «031.752 650s 84 

“Male =——<—s~Sti‘ ; OW”#*#*«éa 74657 8855 650 BSN 

“Ethnicity. —=~=~*S*«SAmeerican Indian/Alaska Native ~—=—=«209.=2S's 717.52 28.91 ©6655 803 

‘Asian t—“‘SéOSOS™;”;”!”!”#ONORSC*#*#«*G7.GO==«-33.58 678 848 

“BlackorAfricanAmerican ~—=—~—-«:191~S«736.93 32.09 650.798 

“Hispanic/latino.  ~~«644.~=«735.01.~=«32.40SssS”~*«<‘ DCS 
Native Hawaiian or Pacific Islander 8 760.38 24.41 702 804 

“Multiple RaceSelected + «72 +~«75468 «23462 +«684.—Sts«B2BCS™ 

“White 12,008 756.44 2667 650 850 
Economic Status” Economically Disadvantaged 1,039 732.16 = 32.31 650 817 
a eens English Learner (EL) 172 709.35 2481 650 783 
Non English Learner 2,048 747.97 31.11 650 839 

“Disabilities -—~—=«sStudents with Disabilities (SWD) +~«=—«.149.=«719.26=««35.86 650 822 
Students without Disabilities 2,093 747.34 31.62 650 820 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). 
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Table A.12.47 Subgroup Performance for Mathematics Scale Scores: Integrated Mathematics III 


Group Type Group N Mean SD Min Max 
“FullSummative Score 568 722.72 44.51 650 850 
“Gender =—st—<‘<‘ésemale”*™=™=©™~™U™~”: 27A4«~*«2048 «45.44 650085 

“Male =—t—<i—si‘sSOS:~:”::””””C‘ MH: 480) SOC 
‘Asiantt—<“‘éSOSOSOS™S*C;*;*;*;*”;”!”!”!”!”*##~C~*~«=769«0O~«&«S.«S2)CSOC*«SCCSSC‘(«;s 
Black or African American 6 710.83 = 15.01 650 824 
Hispanic/Latino 209 694.69 33.70 650 850 
Native Hawaiian or Pacific Islander 1 814.00 n/r 699 699 
Multiple Race Selected 13 749.38 50.97 650 809 
“White 223 756.38 33.22 650 850 | 
Economic Status” Economically Disadvantaged 377,700.30 32.13 650 837 
a eens English Learner (EL) 39 676.90 20.75 650 805 
Non English Learner 529 726.10 43.93 650 850 
“Disabilities -—~—=«sStudents with Disabilities (SWD) +«=«50~==«687.94 36.87 650. 805 
Students without Disabilities 518 726.08 43.76 650 821 


Note: “Economic status was based on participation in National School Lunch Program (NSLP): receipt of free or 
reduced-price lunch (FRL). n/r = not reported due to n<25. 


March 27, 2018 Page 394 


PARCC 2017 Technical Report 


Appendix 12.5: Scale Score Cumulative Frequencies 


Table A.12.48 Scale Score Cumulative Frequencies: ELA/L Grade 3 


Score Band Count Percent Cumblative CHMUaHVE 
Count Percent 

650-654 6,344 1.51 6,344 1.51 

655-659 3,275 0.78 9,619 2.29 

660-664 2,525 0.60 12,144 2.89 

665-669 5,950 1.41 18,094 4.30 

670-674 8,899 2.12 26,993 6.42 

675-679 7,765 1.85 34,758 8.27 

680-684 10,115 2.41 44,873 10.67 
685-689 12,166 2.89 57,039 13.56 
690-694 9,284 2.21 66,323 15.77 
695-699 13,791 3.28 80,114 19.05 
700-704 15,155 3.60 95,269 22.66 
705-709 12,562 2.99 107,831 25.64 
710-714 16,942 4.03 124,773 29.67 
715-719 14,859 3.53 139,632 33.21 
720-724 15,231 3.62 154,863 36.83 
725-729 21,692 5.16 176,555 41.99 
730-734 18,616 4.43 195,171 46.41 
735-739 17,864 4.25 213,035 50.66 
740-744 21,166 5.03 234,201 55.70 
745-749 17,846 4.24 252,047 59.94 
750-754 18,501 4.40 270,548 64.34 
755-759 17,453 4.15 288,001 68.49 
760-764 18,065 4.30 306,066 72.79 
765-769 18,572 4.42 324,638 77.20 
770-774 13,993 3.33 338,631 80.53 
775-779 12,853 3.06 351,484 83.59 
780-784 12,441 2.96 363,925 86.55 
785-789 11,907 2.83 375,832 89.38 
790-794 8,934 2.12 384,766 91.50 
795-799 7,441 1.77 392,207 93.27 
800-804 6,254 1.49 398,461 94.76 
805-809 4,478 1.06 402,939 95.82 
810-814 4,345 1.03 407,284 96.86 
815-819 3,052 0.73 410,336 97.58 
820-824 2,821 0.67 413,157 98.25 
825-829 1,645 0.39 414,802 98.65 
830-834 1,781 0.42 416,583 99.07 
835-839 1,052 0.25 417,635 99.32 
840-844 782 0.19 418,417 99.51 
845-850 2,079 0.49 420,496 100.00 
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Table A.12.49 Scale Score Cumulative Frequencies: ELA/L Grade 4 


Score Band Count Percent CUM TIVE Cumulative 
Count Percent 
650-654 2,498 0.59 2,498 0.59 
655-659 1,164 0.28 3,662 0.87 
660-664 2,040 0.48 5,702 1.35 
665-669 2,432 0.57 8,134 1.92 
670-674 3,870 0.91 12,004 2.84 
675-679 7,679 1.81 19,683 4.65 
680-684 6,257 1.48 25,940 6.13 
685-689 7,893 1.87 33,833 8.00 
690-694 9,126 2.16 42,959 10.15 
695-699 10,847 2.56 53,806 12.72 
700-704 11,192 2.65 64,998 15.36 
705-709 14,967 3.54 79,965 18.90 
710-714 15,778 3.73 95,743 22.63 
715-719 19,251 4.55 114,994 27.18 
720-724 16,011 3.78 131,005 30.96 
725-729 21,843 5.16 152,848 36.13 
730-734 22,392 5.29 175,240 41.42 
735-739 20,655 4.88 195,895 46.30 
740-744 24,517 5.79 220,412 52.09 
745-749 22,186 5.24 242,598 57.34 
750-754 22,063 5.21 264,661 62.55 
755-759 23,471 5.55 288,132 68.10 
760-764 20,070 4.74 308,202 72.84 
765-769 20,332 4.81 328,534 77.65 
770-774 15,115 3.57 343,649 81.22 
775-779 15,583 3.68 359,232 84.91 
780-784 13,333 3.15 372,565 88.06 
785-789 12,688 3.00 385,253 91.06 
790-794 8,667 2.05 393,920 93.10 
795-799 7,970 1.88 401,890 94.99 
800-804 5,154 1.22 407,044 96.21 
805-809 4,724 1.12 411,768 97.32 
810-814 3,440 0.81 415,208 98.14 
815-819 2,114 0.50 417,322 98.63 
820-824 1,986 0.47 419,308 99.10 
825-829 1,140 0.27 420,448 99.37 
830-834 956 0.23 421,404 99.60 
835-839 571 0.13 421,975 99.73 
840-844 391 0.09 422,366 99.83 
845-850 732 0.17 423,098 100.00 
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Table A.12.50 Scale Score Cumulative Frequencies: ELA/L Grade 5 


Score Band Count Percent CUMIN TIVE Cumulative 
Count Percent 

650-654 2,189 0.53 2,189 0.53 
655-659 626 0.15 2,815 0.68 
660-664 1,702 0.41 4,517 1.09 
665-669 2,139 0.52 6,656 1.61 
670-674 3,883 0.94 10,539 2.54 
675-679 5,212 1.26 15,751 3.80 
680-684 6,326 1.53 22,077 5.33 
685-689 7,087 1.71 29,164 7.04 
690-694 7,428 1.79 36,592 8.83 
695-699 12,672 3.06 49,264 11.89 
700-704 10,565 2.55 59,829 14.44 
705-709 14,167 3.42 73,996 17.85 
710-714 17,166 4.14 91,162 22.00 
715-719 14,930 3.60 106,092 25.60 
720-724 19,687 4.75 125,779 30.35 
725-729 17,797 4.29 143,576 34.64 
730-734 22,381 5.40 165,957 40.04 
735-739 22,746 5.49 188,703 45.53 
740-744 22,079 5.33 210,782 50.86 
745-749 22,264 5.37 233,046 56.23 
750-754 24,559 5.93 257,605 62.15 
755-759 22,962 5.54 280,567 67.69 
760-764 20,162 4.86 300,729 72.56 
765-769 20,512 4.95 321,241 77.51 
770-774 17,019 4.11 338,260 81.61 
775-779 15,418 3.72 353,678 85.33 
780-784 13,406 3.23 367,084 88.57 
785-789 11,361 2.74 378,445 91.31 
790-794 8,469 2.04 386,914 93.35 
795-799 7,358 1.78 394,272 95.13 
800-804 6,370 1.54 400,642 96.67 
805-809 3,762 0.91 404,404 97.57 
810-814 3,437 0.83 407,841 98.40 
815-819 1,919 0.46 409,760 98.87 
820-824 1,617 0.39 411,377 99.26 
825-829 1,227 0.30 412,604 99.55 
830-834 704 0.17 413,308 99.72 
835-839 454 0.11 413,762 99.83 
840-844 318 0.08 414,080 99.91 
845-850 381 0.09 414,461 100.00 
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Table A.12.51 Scale Score Cumulative Frequencies: ELA/L Grade 6 


Score Band Count Percent CUMIIATIVE Cumulative 
Count Percent 

650-654 486 0.12 486 0.12 
655-659 88 0.02 574 0.14 
660-664 653 0.16 1,227 0.30 
665-669 1,244 0.31 2,471 0.61 
670-674 1,974 0.49 4,445 1.09 
675-679 2,854 0.70 7,299 1.79 
680-684 4,930 1.21 12,229 3.01 
685-689 6,483 1.59 18,712 4.60 
690-694 11,499 2.83 30,211 7.43 
695-699 12,641 3.11 42,852 10.54 
700-704 14,501 3.57 57,353 14.10 
705-709 13,931 3.43 71,284 17.53 
710-714 17,196 4.23 88,480 21.76 
715-719 18,920 4.65 107,400 26.41 
720-724 20,197 4.97 127,597 31.37 
725-729 21,637 5.32 149,234 36.69 
730-734 21,737 5.34 170,971 42.04 
735-739 23,286 5.73 194,257 47.76 
740-744 24,782 6.09 219,039 53.86 
745-749 24,506 6.03 243,545 59.88 
750-754 22,416 5.51 265,961 65.39 
755-759 22,949 5.64 288,910 71.04 
760-764 20,482 5.04 309,392 76.07 
765-769 17,547 4.31 326,939 80.39 
770-774 17,475 4.30 344,414 84.68 
775-779 14,109 3.47 358,523 88.15 
780-784 12,367 3.04 370,890 91.19 
785-789 9,340 2.30 380,230 93.49 
790-794 7,177 1.76 387,407 95.26 
795-799 5,271 1.30 392,678 96.55 
800-804 4,086 1.00 396,764 97.56 
805-809 3,174 0.78 399,938 98.34 
810-814 2,420 0.60 402,358 98.93 
815-819 1,587 0.39 403,945 99.32 
820-824 871 0.21 404,816 99.54 
825-829 669 0.16 405,485 99.70 
830-834 519 0.13 406,004 99.83 
835-839 229 0.06 406,233 99.88 
840-844 173 0.04 406,406 99.93 
845-850 299 0.07 406,705 100.00 
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Table A.12.52 Scale Score Cumulative Frequencies: ELA/L Grade 7 


Score Band Count Percent CUM RIVE Cumulative 
Count Percent 

650-654 2,823 0.69 2,823 0.69 

655-659 1,127 0.28 3,950 0.97 

660-664 2,537 0.62 6,487 1.60 

665-669 4,318 1.06 10,805 2.66 

670-674 5,902 1.45 16,707 4.11 

675-679 6,356 1.56 23,063 5.68 

680-684 5,171 1.27 28,234 6.95 

685-689 9,929 2.44 38,163 9.40 

690-694 8,318 2.05 46,481 11.44 
695-699 11,201 2.76 57,682 14.20 
700-704 9,895 2.44 67,577 16.64 
705-709 14,247 3.51 81,824 20.14 
710-714 11,375 2.80 93,199 22.94 
715-719 16,295 4.01 109,494 26.96 

720-724 15,391 3.79 124,885 30.74 
725-729 20,723 5.10 145,608 35.85 
730-734 16,140 3.97 161,748 39.82 
735-739 19,884 4.90 181,632 44.71 
740-744 21,781 5.36 203,413 50.08 
745-749 20,408 5.02 223,821 55.10 
750-754 21,562 5.31 245,383 60.41 
755-759 21,289 5.24 266,672 65.65 
760-764 17,377 4.28 284,049 69.93 
765-769 19,876 4.89 303,925 74.82 
770-774 16,837 4.14 320,762 78.97 
775-779 14,186 3.49 334,948 82.46 
780-784 14,202 3.50 349,150 85.95 
785-789 11,493 2.83 360,643 88.78 
790-794 9,964 2.45 370,607 91.24 
795-799 8,094 1.99 378,701 93.23 
800-804 5,814 1.43 384,515 94.66 
805-809 5,666 1.39 390,181 96.06 
810-814 4,702 1.16 394,883 97.21 
815-819 2,951 0.73 397,834 97.94 
820-824 2,415 0.59 400,249 98.53 
825-829 1,733 0.43 401,982 98.96 
830-834 1,289 0.32 403,271 99.28 
835-839 1,004 0.25 404,275 99.53 
840-844 679 0.17 404,954 99.69 
845-850 1,247 0.31 406,201 100.00 
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Table A.12.53 Scale Score Cumulative Frequencies: ELA/L Grade 8 


Score Band Count Percent CUMUUTE RIVE Cumulative 
Count Percent 

650-654 2,647 0.66 2,647 0.66 

655-659 1,871 0.47 4,518 1.13 

660-664 2,429 0.61 6,947 1.74 

665-669 4,408 1.10 11,355 2.84 

670-674 4,019 1.01 15,374 3.85 

675-679 7,715 1.93 23,089 5.78 

680-684 6,799 1.70 29,888 7.48 

685-689 10,429 2.61 40,317 10.10 
690-694 9,922 2.48 50,239 12.58 

695-699 10,204 2.56 60,443 15.13 

700-704 12,625 3.16 73,068 18.30 
705-709 12,580 3.15 85,648 21.45 

710-714 14,264 3.57 99,912 25.02 

715-719 14,755 3.69 114,667 28.71 

720-724 16,363 4.10 131,030 32.81 
725-729 17,160 4.30 148,190 37.11 
730-734 20,666 5.17 168,856 42.28 
735-739 19,262 4.82 188,118 47.10 
740-744 19,133 4.79 207,251 51.89 
745-749 19,354 4.85 226,605 56.74 
750-754 19,075 4.78 245,680 61.52 
755-759 20,445 5.12 266,125 66.64 
760-764 18,101 4.53 284,226 71.17 
765-769 16,095 4.03 300,321 75.20 
770-774 13,692 3.43 314,013 78.63 
775-779 15,765 3.95 329,778 82.57 
780-784 12,989 3.25 342,767 85.83 
785-789 10,618 2.66 353,385 88.49 
790-794 9,480 2.37 362,865 90.86 
795-799 7,487 1.87 370,352 92.73 
800-804 6,336 1.59 376,688 94.32 
805-809 6,157 1.54 382,845 95.86 
810-814 3,940 0.99 386,785 96.85 
815-819 3,261 0.82 390,046 97.67 
820-824 2,451 0.61 392,497 98.28 
825-829 1,918 0.48 394,415 98.76 
830-834 1,364 0.34 395,779 99.10 
835-839 1,015 0.25 396,794 99.35 
840-844 992 0.25 397,786 99.60 
845-850 1,585 0.40 399,371 100.00 
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Table A.12.54 Scale Score Cumulative Frequencies: ELA/L Grade 9 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 1,350 0.72 1,350 0.72 
655-659 638 0.34 1,988 1.06 
660-664 1,471 0.78 3,459 1.84 
665-669 1,973 1.05 5,432 2.89 
670-674 2,319 1.23 7,751 4.12 
675-679 2,817 1.50 10,568 5.62 
680-684 3,917 2.08 14,485 7.70 
685-689 4,900 2.60 19,385 10.30 
690-694 5,611 2.98 24,996 13.28 
695-699 5,432 2.89 30,428 16.17 
700-704 4,871 2.59 35,299 18.76 
705-709 5,710 3.03 41,009 21.79 
710-714 6,250 3.32 47,259 25.11 
715-719 7,179 3.81 54,438 28.93 
720-724 7,223 3.84 61,661 32.76 
725-729 8,955 4.76 70,616 37.52 
730-734 8,169 4.34 78,785 41.86 
735-739 9,928 5.28 88,713 47.14 
740-744 9,927 5.27 98,640 52.41 
745-749 9,716 5.16 108,356 57.57 
750-754 9,684 5.15 118,040 62.72 
755-759 9,302 4.94 127,342 67.66 
760-764 8,095 4.30 135,437 71.96 
765-769 8,986 4.77 144,423 76.74 
770-774 7,979 4.24 152,402 80.98 
775-779 6,554 3.48 158,956 84.46 
780-784 5,735 3.05 164,691 87.51 
785-789 5,109 2.71 169,800 90.22 
790-794 4,000 2.13 173,800 92.35 
795-799 4,026 2.14 177,826 94.49 
800-804 2,515 1.34 180,341 95.82 
805-809 2,138 1.14 182,479 96.96 
810-814 1,481 0.79 183,960 97.75 
815-819 1,229 0.65 185,189 98.40 
820-824 895 0.48 186,084 98.88 
825-829 623 0.33 186,707 99.21 
830-834 524 0.28 187,231 99.48 
835-839 363 0.19 187,594 99.68 
840-844 134 0.07 187,728 99.75 
845-850 473 0.25 188,201 100.00 
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Table A.12.55 Scale Score Cumulative Frequencies: ELA/L Grade 10 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 6,691 3.89 6,691 3.89 
655-659 1,759 1.02 8,450 4.91 
660-664 2,676 1.56 11,126 6.47 
665-669 2,907 1.69 14,033 8.16 
670-674 2,846 1.65 16,879 9.82 
675-679 3,843 2.23 20,722 12.05 
680-684 3,449 2.01 24,171 14.06 
685-689 4,151 2.41 28,322 16.47 
690-694 3,781 2.20 32,103 18.67 
695-699 4,388 2.55 36,491 21.22 
700-704 4,196 2.44 40,687 23.66 
705-709 4,247 2.47 44,934 26.13 
710-714 4,887 2.84 49,821 28.97 
715-719 5,657 3.29 55,478 32.26 
720-724 5,704 3.32 61,182 35.58 
725-729 5,756 3.35 66,938 38.93 
730-734 6,874 4.00 73,812 42.92 
735-739 6,692 3.89 80,504 46.81 
740-744 6,716 3.91 87,220 50.72 
745-749 7,226 4.20 94,446 54.92 
750-754 7,376 4.29 101,822 59.21 
755-759 6,570 3.82 108,392 63.03 
760-764 6,373 3.71 114,765 66.74 
765-769 7,014 4.08 121,779 70.82 
770-774 6,024 3.50 127,803 74.32 
775-779 5,723 3.33 133,526 77.65 
780-784 5,922 3.44 139,448 81.09 
785-789 5,013 2.92 144,461 84.01 
790-794 4,687 2.73 149,148 86.73 
795-799 3,771 2.19 152,919 88.92 
800-804 3,476 2.02 156,395 90.95 
805-809 3,323 1.93 159,718 92.88 
810-814 2,327 1.35 162,045 94.23 
815-819 2,303 1.34 164,348 95.57 
820-824 1,756 1.02 166,104 96.59 
825-829 1,450 0.84 167,554 97.43 
830-834 1,046 0.61 168,600 98.04 
835-839 974 0.57 169,574 98.61 
840-844 580 0.34 170,154 98.95 
845-850 1,811 1.05 171,965 100.00 
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Table A.12.56 Scale Score Cumulative Frequencies: ELA/L Grade 11 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 1,014 0.94 1,014 0.94 
655-659 989 0.92 2,003 1.86 
660-664 553 0.51 2,556 2.38 
665-669 1,679 1.56 4,235 3.94 
670-674 2,146 1.99 6,381 5.93 
675-679 2,445 2.27 8,826 8.20 
680-684 2,598 2.41 11,424 10.62 
685-689 4,539 4.22 15,963 14.84 
690-694 3,333 3.10 19,296 17.93 
695-699 3,291 3.06 22,587 20.99 
700-704 3,524 3.28 26,111 24.27 
705-709 4,070 3.78 30,181 28.05 
710-714 3,670 3.41 33,851 31.46 
715-719 5,217 4.85 39,068 36.31 
720-724 3,316 3.08 42,384 39.39 
725-729 4,898 4.55 47,282 43.95 
730-734 5,233 4.86 52,515 48.81 
735-739 4,616 4.29 57,131 53.10 
740-744 5,408 5.03 62,539 58.13 
745-749 4,593 4.27 67,132 62.39 
750-754 5,415 5.03 72,547 67.43 
755-759 5,007 4.65 77,554 72.08 
760-764 4,756 4.42 82,310 76.50 
765-769 3,345 3.11 85,655 79.61 
770-774 3,995 3.71 89,650 83.32 
775-779 3,442 3.20 93,092 86.52 
780-784 2,660 2.47 95,752 88.99 
785-789 2,534 2.36 98,286 91.35 
790-794 1,988 1.85 100,274 93.20 
795-799 1,696 1.58 101,970 94.77 
800-804 1,396 1.30 103,366 96.07 
805-809 1,106 1.03 104,472 97.10 
810-814 744 0.69 105,216 97.79 
815-819 629 0.58 105,845 98.38 
820-824 531 0.49 106,376 98.87 
825-829 318 0.30 106,694 99.16 
830-834 286 0.27 106,980 99.43 
835-839 172 0.16 107,152 99.59 
840-844 140 0.13 107,292 99.72 
845-850 301 0.28 107,593 100.00 


March 27, 2018 Page 403 


PARCC 2017 Technical Report 


Table A.12.57 Scale Score Cumulative Frequencies: Mathematics Grade 3 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 4,919 1.16 4,919 1.16 

655-659 368 0.09 5,287 1.24 

660-664 3,641 0.86 8,928 2.10 

665-669 2,342 0.55 11,270 2.65 

670-674 4,032 0.95 15,302 3.59 

675-679 5,405 1.27 20,707 4.86 

680-684 7,005 1.65 27,712 6.51 

685-689 7,627 1.79 35,339 8.30 

690-694 9,456 2.22 44,795 10.52 

695-699 9,882 2.32 54,677 12.84 
700-704 12,397 2.91 67,074 15.75 

705-709 14,929 3.51 82,003 19.26 

710-714 14,304 3.36 96,307 22.62 

715-719 17,557 4.12 113,864 26.74 
720-724 19,277 4.53 133,141 31.27 
725-729 20,494 4.81 153,635 36.09 
730-734 21,715 5.10 175,350 41.19 
735-739 24,580 5.77 199,930 46.96 
740-744 19,577 4.60 219,507 51.56 
745-749 25,025 5.88 244,532 57.44 
750-754 20,836 4.89 265,368 62.33 
755-759 23,335 5.48 288,703 67.81 
760-764 18,611 4.37 307,314 72.18 
765-769 21,270 5.00 328,584 77.18 
770-774 16,611 3.90 345,195 81.08 
775-779 13,912 3.27 359,107 84.35 
780-784 14,260 3.35 373,367 87.70 
785-789 9,815 2.31 383,182 90.00 
790-794 9,409 2.21 392,591 92.21 
795-799 9,208 2.16 401,799 94.38 
800-804 4,167 0.98 405,966 95.35 
805-809 4,846 1.14 410,812 96.49 
810-814 3,894 0.91 414,706 97.41 
815-819 2,649 0.62 417,355 98.03 
820-824 2,412 0.57 419,767 98.60 
825-829 1,250 0.29 421,017 98.89 
830-834 1,210 0.28 422,227 99.17 
835-839 1,182 0.28 423,409 99.45 
840-844 414 0.10 423,823 99.55 
845-850 1,921 0.45 425,744 100.00 
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Table A.12.58 Scale Score Cumulative Frequencies: Mathematics Grade 4 


Score Band Count Percent CuMUIATIVE Cumulative 
Count Percent 

650-654 2,110 0.49 2,110 0.49 

655-659 739 0.17 2,849 0.67 

660-664 2,096 0.49 4,945 1.16 

665-669 1,283 0.30 6,228 1.46 

670-674 4,522 1.06 10,750 2.52 

675-679 6,159 1.44 16,909 3.96 

680-684 7,257 1.70 24,166 5.66 

685-689 8,555 2.01 32,721 7.67 

690-694 10,844 2.54 43,565 10.21 
695-699 15,211 3.57 58,776 13.78 
700-704 15,503 3.63 74,279 17.41 
705-709 19,607 4.60 93,886 22.01 
710-714 21,485 5.04 115,371 27.04 
715-719 20,576 4.82 135,947 31.87 
720-724 20,356 4.77 156,303 36.64 
725-729 25,095 5.88 181,398 42.52 
730-734 21,868 5.13 203,266 47.65 
735-739 21,562 5.05 224,828 52.70 
740-744 26,322 6.17 251,150 58.87 
745-749 22,451 5.26 273,601 64.13 
750-754 21,513 5.04 295,114 69.18 
755-759 21,936 5.14 317,050 74.32 
760-764 16,895 3.96 333,945 78.28 
765-769 18,638 4.37 352,583 82.65 
770-774 15,120 3.54 367,703 86.19 
775-779 13,694 3.21 381,397 89.40 
780-784 10,653 2.50 392,050 91.90 
785-789 9,082 2.13 401,132 94.03 
790-794 7,609 1.78 408,741 95.81 
795-799 3,833 0.90 412,574 96.71 
800-804 4,813 1.13 417,387 97.84 
805-809 2,306 0.54 419,693 98.38 
810-814 1,940 0.45 421,633 98.83 
815-819 1,664 0.39 423,297 99.22 
820-824 1,223 0.29 424,520 99.51 
825-829 573 0.13 425,093 99.64 
830-834 485 0.11 425,578 99.76 
835-839 387 0.09 425,965 99.85 
840-844 118 0.03 426,083 99.88 
845-850 526 0.12 426,609 100.00 
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Table A.12.59 Scale Score Cumulative Frequencies: Mathematics Grade 5 


Score Band Count Percent CuMUTaTIVE Cumulative 
Count Percent 

650-654 1,786 0.43 1,786 0.43 
655-659 454 0.11 2,240 0.54 
660-664 2,025 0.49 4,265 1.02 
665-669 2,059 0.49 6,324 1.52 
670-674 3,509 0.84 9,833 2.36 
675-679 1,665 0.40 11,498 2.76 
680-684 6,707 1.61 18,205 4.37 
685-689 8,824 2.12 27,029 6.49 
690-694 9,949 2.39 36,978 8.88 
695-699 12,735 3.06 49,713 11.93 
700-704 19,901 4.78 69,614 16.71 
705-709 13,443 3.23 83,057 19.94 
710-714 25,093 6.02 108,150 25.96 
715-719 15,553 3.73 123,703 29.70 
720-724 24,591 5.90 148,294 35.60 
725-729 23,517 5.65 171,811 41.25 
730-734 27,578 6.62 199,389 47.87 
735-739 24,470 5.87 223,859 53.74 
740-744 23,128 5.55 246,987 59.29 
745-749 24,611 5.91 271,598 65.20 
750-754 20,303 4.87 291,901 70.07 
755-759 20,964 5.03 312,865 75.11 
760-764 18,417 4.42 331,282 79.53 
765-769 16,537 3.97 347,819 83.50 
770-774 14,390 3.45 362,209 86.95 
775-779 12,516 3.00 374,725 89.96 
780-784 10,386 2.49 385,111 92.45 
785-789 8,402 2.02 393,513 94.47 
790-794 6,177 1.48 399,690 95.95 
795-799 5,041 1.21 404,731 97.16 
800-804 2,931 0.70 407,662 97.86 
805-809 3,499 0.84 411,161 98.70 
810-814 1,313 0.32 412,474 99.02 
815-819 1,114 0.27 413,588 99.29 
820-824 921 0.22 414,509 99.51 
825-829 729 0.18 415,238 99.68 
830-834 158 0.04 415,396 99.72 
835-839 494 0.12 415,890 99.84 
840-844 162 0.04 416,052 99.88 
845-850 504 0.12 416,556 100.00 
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Table A.12.60 Scale Score Cumulative Frequencies: Mathematics Grade 6 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 2,933 0.72 2,933 0.72 

655-659 1,601 0.39 4,534 1.11 

660-664 830 0.20 5,364 1.31 

665-669 4,188 1.02 9,552 2.34 

670-674 1,435 0.35 10,987 2.69 

675-679 6,467 1.58 17,454 4.27 

680-684 8,244 2.02 25,698 6.29 

685-689 9,123 2.23 34,821 8.52 

690-694 7,284 1.78 42,105 10.30 
695-699 18,829 4.61 60,934 14.91 
700-704 18,958 4.64 79,892 19.55 
705-709 14,600 3.57 94,492 23.12 
710-714 19,661 4.81 114,153 27.93 
715-719 20,867 5.11 135,020 33.04 
720-724 25,068 6.13 160,088 39.17 
725-729 23,535 5.76 183,623 44.93 
730-734 22,296 5.46 205,919 50.39 
735-739 27,786 6.80 233,705 57.19 
740-744 19,169 4.69 252,874 61.88 
745-749 23,664 5.79 276,538 67.67 
750-754 21,303 5.21 297,841 72.88 
755-759 17,095 4.18 314,936 77.06 
760-764 18,878 4.62 333,814 81.68 
765-769 16,208 3.97 350,022 85.65 
770-774 13,977 3.42 363,999 89.07 
775-779 10,279 2.52 374,278 91.59 
780-784 9,027 2.21 383,305 93.79 
785-789 6,885 1.68 390,190 95.48 
790-794 5,608 1.37 395,798 96.85 
795-799 4,216 1.03 400,014 97.88 
800-804 2,478 0.61 402,492 98.49 
805-809 1,532 0.37 404,024 98.86 
810-814 1,272 0.31 405,296 99.18 
815-819 1,050 0.26 406,346 99.43 
820-824 843 0.21 407,189 99.64 
825-829 504 0.12 407,693 99.76 
830-834 170 0.04 407,863 99.80 
835-839 123 0.03 407,986 99.83 
840-844 212 0.05 408,198 99.89 
845-850 466 0.11 408,664 100.00 
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Table A.12.61 Scale Score Cumulative Frequencies: Mathematics Grade 7 


Score Band Count Percent CUMUIATIVE Cumulative 
Count Percent 

650-654 2,080 0.53 2,080 0.53 

655-659 659 0.17 2,739 0.70 

660-664 14 0.00 2,753 0.70 

665-669 1,650 0.42 4,403 1.12 

670-674 3,921 1.00 8,324 2.11 

675-679 123 0.03 8,447 2.14 

680-684 9,863 2.50 18,310 4.65 

685-689 2,907 0.74 21,217 5.39 

690-694 11,344 2.88 32,561 8.27 

695-699 13,002 3.30 45,563 11.57 
700-704 17,183 4.36 62,746 15.93 
705-709 17,150 4.35 79,896 20.28 
710-714 20,563 5.22 100,459 25.50 
715-719 26,519 6.73 126,978 32.23 
720-724 20,989 5.33 147,967 37.56 
725-729 25,097 6.37 173,064 43.93 
730-734 27,844 7.07 200,908 51.00 
735-739 28,593 7.26 229,501 58.26 
740-744 25,400 6.45 254,901 64.71 
745-749 25,198 6.40 280,099 71.10 
750-754 21,496 5.46 301,595 76.56 
755-759 18,765 4.76 320,360 81.32 
760-764 15,776 4.00 336,136 85.33 
765-769 15,805 4.01 351,941 89.34 
770-774 11,751 2.98 363,692 92.32 
775-779 8,810 2.24 372,502 94.56 
780-784 6,187 1.57 378,689 96.13 
785-789 5,689 1.44 384,378 97.57 
790-794 3,418 0.87 387,796 98.44 
795-799 2,174 0.55 389,970 98.99 
800-804 1,438 0.37 391,408 99.36 
805-809 854 0.22 392,262 99.57 
810-814 656 0.17 392,918 99.74 
815-819 399 0.10 393,317 99.84 
820-824 198 0.05 393,515 99.89 
825-829 131 0.03 393,646 99.92 
830-834 84 0.02 393,730 99.95 
835-839 86 0.02 393,816 99.97 
840-844 21 0.01 393,837 99.97 
845-850 105 0.03 393,942 100.00 
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Table A.12.62 Scale Score Cumulative Frequencies: Mathematics Grade 8 


Score Band Count Percent CuMUTaTIve Cumulative 
Count Percent 

650-654 7,653 2.43 7,653 2.43 

655-659 1,401 0.45 9,054 2.88 

660-664 2,915 0.93 11,969 3.81 

665-669 5,864 1.86 17,833 5.67 

670-674 4,605 1.46 22,438 7.13 

675-679 7,565 2.41 30,003 9.54 

680-684 11,739 3.73 41,742 13.27 
685-689 12,192 3.88 53,934 17.15 
690-694 15,969 5.08 69,903 22.23 
695-699 15,853 5.04 85,756 27.27 
700-704 9,156 2.91 94,912 30.18 
705-709 18,981 6.04 113,893 36.22 
710-714 15,569 4.95 129,462 41.17 
715-719 13,272 4.22 142,734 45.39 
720-724 13,119 4.17 155,853 49.56 
725-729 20,134 6.40 175,987 55.96 
730-734 13,302 4.23 189,289 60.19 
735-739 15,770 5.01 205,059 65.20 
740-744 13,197 4.20 218,256 69.40 
745-749 14,913 4.74 233,169 74.14 
750-754 12,753 4.06 245,922 78.20 
755-759 10,956 3.48 256,878 81.68 
760-764 9,358 2.98 266,236 84.66 
765-769 10,002 3.18 276,238 87.84 
770-774 6,806 2.16 283,044 90.00 
775-779 7,185 2.28 290,229 92.29 
780-784 5,200 1.65 295,429 93.94 
785-789 4,637 1.47 300,066 95.42 
790-794 3,325 1.06 303,391 96.47 
795-799 2,730 0.87 306,121 97.34 
800-804 1,876 0.60 307,997 97.94 
805-809 1,631 0.52 309,628 98.46 
810-814 1,296 0.41 310,924 98.87 
815-819 850 0.27 311,774 99.14 
820-824 563 0.18 312,337 99.32 
825-829 575 0.18 312,912 99.50 
830-834 457 0.15 313,369 99.65 
835-839 244 0.08 313,613 99.72 
840-844 187 0.06 313,800 99.78 
845-850 684 0.22 314,484 100.00 
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Table A.12.63 Scale Score Cumulative Frequencies: Algebra | 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 1,681 0.64 1,681 0.64 
655-659 335 0.13 2,016 0.77 
660-664 705 0.27 2,721 1.04 
665-669 2,358 0.90 5,079 1.95 
670-674 626 0.24 5,705 2.19 
675-679 2,065 0.79 7,770 2.98 
680-684 4,804 1.84 12,574 4.82 
685-689 3,237 1.24 15,811 6.06 
690-694 7,224 2.77 23,035 8.83 
695-699 13,251 5.08 36,286 13.91 
700-704 6,576 2.52 42,862 16.43 
705-709 15,113 5.79 57,975 22.23 
710-714 10,102 3.87 68,077 26.10 
715-719 18,239 6.99 86,316 33.10 
720-724 15,447 5.92 101,763 39.02 
725-729 10,666 4.09 112,429 43.11 
730-734 14,470 5.55 126,899 48.66 
735-739 15,006 5.75 141,905 54.41 
740-744 12,893 4.94 154,798 59.35 
745-749 11,913 4.57 166,711 63.92 
750-754 10,314 3.95 177,025 67.88 
755-759 11,867 4.55 188,892 72.43 
760-764 10,428 4.00 199,320 76.42 
765-769 10,080 3.86 209,400 80.29 
770-774 10,342 3.97 219,742 84.25 
775-779 7,261 2.78 227,003 87.04 
780-784 7,447 2.86 234,450 89.89 
785-789 5,947 2.28 240,397 92.17 
790-794 4,690 1.80 245,087 93.97 
795-799 4,013 1.54 249,100 95.51 
800-804 3,198 1.23 252,298 96.74 
805-809 2,334 0.89 254,632 97.63 
810-814 1,869 0.72 256,501 98.35 
815-819 995 0.38 257,496 98.73 
820-824 911 0.35 258,407 99.08 
825-829 515 0.20 258,922 99.28 
830-834 601 0.23 259,523 99.51 
835-839 356 0.14 259,879 99.64 
840-844 205 0.08 260,084 99.72 
845-850 722 0.28 260,806 100.00 
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Table A.12.64 Scale Score Cumulative Frequencies: Geometry 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 468 0.34 468 0.34 
655-659 161 0.12 629 0.46 
660-664 1 0.00 630 0.46 
665-669 396 0.29 1,026 0.75 
670-674 942 0.68 1,968 1.43 
675-679 230 0.17 2,198 1.60 
680-684 1,270 0.92 3,468 2.52 
685-689 2,620 1.91 6,088 4.43 
690-694 2,994 2.18 9,082 6.60 
695-699 3,504 2.55 12,586 9.15 
700-704 6,500 4.73 19,086 13.88 
705-709 6,842 4.97 25,928 18.85 
710-714 8,893 6.47 34,821 25.32 
715-719 9,486 6.90 44,307 32.22 
720-724 6,914 5.03 51,221 37.24 
725-729 9,035 6.57 60,256 43.81 
730-734 9,225 6.71 69,481 50.52 
735-739 7,013 5.10 76,494 55.62 
740-744 9,047 6.58 85,541 62.20 
745-749 8,143 5.92 93,684 68.12 
750-754 8,457 6.15 102,141 74.27 
755-759 7,245 5.27 109,386 79.54 
760-764 6,226 4.53 115,612 84.06 
765-769 6,005 4.37 121,617 88.43 
770-774 5,005 3.64 126,622 92.07 
775-779 3,890 2.83 130,512 94.90 
780-784 2,369 1.72 132,881 96.62 
785-789 2,038 1.48 134,919 98.10 
790-794 1,153 0.84 136,072 98.94 
795-799 626 0.46 136,698 99.40 
800-804 352 0.26 137,050 99.65 
805-809 195 0.14 137,245 99.79 
810-814 117 0.09 137,362 99.88 
815-819 65 0.05 137,427 99.93 
820-824 52 0.04 137,479 99.96 
825-829 15 0.01 137,494 99.97 
830-834 1 0.00 137,495 99.97 
835-839 15 0.01 137,510 99.99 
840-844 9 0.01 137,519 99.99 
845-850 11 0.01 137,530 100.00 
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Table A.12.65 Scale Score Cumulative Frequencies: Algebra II 


Score Band Count Percent Cumulative Cumulative 
Count Percent 

650-654 3,415 2.78 3,415 2.78 

655-659 1,256 1.02 4,671 3.81 

660-664 2,402 1.96 7,073 5.76 

665-669 2,293 1.87 9,366 7.63 

670-674 3,096 2.52 12,462 10.16 
675-679 4,656 3.79 17,118 13.95 
680-684 2,931 2.39 20,049 16.34 
685-689 6,369 5.19 26,418 21.53 
690-694 7,262 5.92 33,680 27.45 
695-699 6,989 5.70 40,669 33.14 
700-704 6,132 5.00 46,801 38.14 
705-709 5,435 4.43 52,236 42.57 
710-714 4,879 3.98 57,115 46.54 
715-719 4,545 3.70 61,660 50.25 
720-724 5,963 4.86 67,623 55.11 
725-729 4,399 3.58 72,022 58.69 
730-734 5,690 4.64 77,712 63.33 
735-739 2,963 2.41 80,675 65.74 
740-744 4,888 3.98 85,563 69.73 
745-749 4,388 3.58 89,951 73.30 
750-754 4,424 3.61 94,375 76.91 
755-759 3,561 2.90 97,936 79.81 
760-764 3,146 2.56 101,082 82.37 
765-769 3,187 2.60 104,269 84.97 
770-774 3,626 2.95 107,895 87.93 
775-779 2,179 1.78 110,074 89.70 
780-784 2,722 2.22 112,796 91.92 
785-789 1,817 1.48 114,613 93.40 
790-794 1,741 1.42 116,354 94.82 
795-799 1,301 1.06 117,655 95.88 
800-804 1,267 1.03 118,922 96.91 
805-809 823 0.67 119,745 97.58 
810-814 731 0.60 120,476 98.18 
815-819 530 0.43 121,006 98.61 
820-824 363 0.30 121,369 98.91 
825-829 389 0.32 121,758 99.22 
830-834 225 0.18 121,983 99.41 
835-839 221 0.18 122,204 99.59 
840-844 122 0.10 122,326 99.69 
845-850 384 0.31 122,710 100.00 
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Table A.12.66 Scale Score Cumulative Frequencies: Integrated Mathematics | 


Score Band Count Percent CuMUTATIVE Cumulative 
Count Percent 

650-654 35 0.32 35 0.32 

655-659 61 0.56 96 0.88 

660-664 21 0.19 117 1.07 

665-669 155 1.41 272 2.48 

670-674 22 0.20 294 2.68 

675-679 235 2.14 529 4.82 

680-684 34 0.31 563 5.13 

685-689 406 3.70 969 8.84 

690-694 532 4.85 1,501 13.69 
695-699 553 5.04 2,054 18.73 
700-704 593 5.41 2,647 24.14 
705-709 554 5.05 3,201 29.20 
710-714 549 5.01 3,750 34.20 
715-719 898 8.19 4,648 42.39 
720-724 394 3.59 5,042 45.99 
725-729 740 6.75 5,782 52.74 
730-734 560 5.11 6,342 57.84 
735-739 565 5.15 6,907 63.00 
740-744 451 4.11 7,358 67.11 
745-749 593 5.41 7,951 72.52 
750-754 363 3.31 8,314 75.83 
755-759 482 4.40 8,796 80.23 
760-764 421 3.84 9,217 84.07 
765-769 255 2.33 9,472 86.39 
770-774 326 2.97 9,798 89.37 
775-779 262 2.39 10,060 91.75 
780-784 206 1.88 10,266 93.63 
785-789 136 1.24 10,402 94.87 
790-794 164 1.50 10,566 96.37 
795-799 72 0.66 10,638 97.03 
800-804 67 0.61 10,705 97.64 
805-809 63 0.57 10,768 98.21 
810-814 43 0.39 10,811 98.60 
815-819 31 0.28 10,842 98.89 
820-824 33 0.30 10,875 99.19 
825-829 29 0.26 10,904 99.45 
830-834 20 0.18 10,924 99.64 
835-839 16 0.15 10,940 99.78 
840-844 4 0.04 10,944 99.82 
845-850 20 0.18 10,964 100.00 
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Table A.12.67 Scale Score Cumulative Frequencies: Integrated Mathematics II 


Score Band Count Percent CUMUTATIVE Cumulative 
Count Percent 

650-654 2 0.09 2 0.09 

655-659 10 0.45 12 0.53 

660-664 0 0.00 12 0.53 

665-669 14 0.62 26 1.16 

670-674 0 0.00 26 1.16 

675-679 23 1.03 49 2.18 

680-684 21 0.94 70 3.12 

685-689 15 0.67 85 3.79 

690-694 55 2.45 140 6.24 

695-699 59 2.63 199 8.87 

700-704 63 2.81 262 11.68 
705-709 92 4.10 354 15.78 
710-714 79 3.52 433 19.30 
715-719 80 3.57 513 22.87 
720-724 90 4.01 603 26.88 
725-729 153 6.82 756 33.70 
730-734 76 3.39 832 37.09 
735-739 62 2.76 894 39.86 
740-744 150 6.69 1,044 46.54 
745-749 171 7.62 1,215 54.17 
750-754 74 3.30 1,289 57.47 
755-759 125 5.57 1,414 63.04 
760-764 163 7.27 1,577 70.31 
765-769 107 4.77 1,684 75.08 
770-774 147 6.55 1,831 81.63 
775-779 101 4.50 1,932 86.13 
780-784 95 4.24 2,027 90.37 
785-789 57 2.54 2,084 92.91 
790-794 41 1.83 2,125 94.74 
795-799 22 0.98 2,147 95.72 
800-804 39 1.74 2,186 97.46 
805-809 14 0.62 2,200 98.08 
810-814 17 0.76 2,217 98.84 
815-819 2 0.09 2,219 98.93 
820-824 6 0.27 2,225 99.20 
825-829 7 0.31 2,232 99.51 
830-834 0 0.00 2,232 99.51 
835-839 4 0.18 2,236 99.69 
840-844 2 0.09 2,238 99.78 
845-850 5 0.22 2,243 100.00 
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Table A.12.68 Scale Score Cumulative Frequencies: Integrated Mathematics III 


Score Band Count Percent CUTTS Cumulative 
Count Percent 

650-654 28 4.93 28 4.93 
655-659 0 0.00 28 4.93 
660-664 26 4.58 54 9.51 
665-669 0 0.00 54 9.51 
670-674 31 5.46 85 14.96 
675-679 32 5.63 117 20.60 
680-684 2 0.35 119 20.95 
685-689 51 8.98 170 29.93 
690-694 28 4.93 198 34.86 
695-699 27 4.75 225 39.61 
700-704 0 0.00 225 39.61 
705-709 30 5.28 255 44.89 
710-714 19 3.35 274 48.24 
715-719 19 3.35 293 51.58 
720-724 11 1.94 304 53.52 
725-729 27 4.75 331 58.27 
730-734 13 2.29 344 60.56 
735-739 12 2.11 356 62.68 
740-744 19 3.35 375 66.02 
745-749 19 3.35 394 69.37 
750-754 22 3.87 416 73.24 
755-759 11 1.94 427 75.18 
760-764 21 3.70 448 78.87 
765-769 10 1.76 458 80.63 
770-774 23 4.05 481 84.68 
775-779 16 2.82 497 87.50 
780-784 11 1.94 508 89.44 
785-789 14 2.46 522 91.90 
790-794 17 2.99 539 94.89 
795-799 8 1.41 547 96.30 
800-804 6 1.06 553 97.36 
805-809 4 0.70 557 98.06 
810-814 1 0.18 558 98.24 
815-819 2 0.35 560 98.59 
820-824 3 0.53 563 99.12 
825-829 2 0.35 565 99.47 
830-834 0 0.00 565 99.47 
835-839 1 0.18 566 99.65 
840-844 1 0.18 567 99.82 
845-850 1 0.18 568 100.00 
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Appendix 13: Growth 


Appendix 13 provides the summary growth results for subgroups for grade 4 — 11 ELA/L and 
mathematics 3 — 8 and high school. Integrated mathematics II and III do not have sufficient sample sizes 
for subgroup summary analysis. 


Table A.13.1 Summary of SGP Estimates for Subgroups: Grade 4 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 203,109 48.22 13.13 47 

Female 195,771 51.83 12.97 53 
Ethnicity 

White 174,974 52.27 12.88 53 

African American 67,347 44.70 13.24 43 

Asian/Pacific Islander 23,183 59.11 12.56 63 

American Indian/Alaska Native 5,114 41.26 13.36 38 

Hispanic 113,794 48.13 13.29 47 

Multiple 13,626 49.88 13.00 50 
Special Instructional Needs 

Economically Disadvantaged 195,933 46.12 13.27 45 

Not-economically Disadvantaged 202,947 53.72 12.84 55 

English Learner (EL) 34,649 45.78 13.91 44 

Non English Learner 364,231 50.39 12.97 51 

Students with Disabilities (SWD) 62,933 42.50 13.56 39 

Students without Disabilities 335,947 51.39 12.96 52 
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Table A.13.2 Summary of SGP Estimates for Subgroups: Grade 5 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 199,403 47.81 14.40 47 

Eamiaté 191,991 52.38 14.12 53 
Ethnicity 

White 174,782 50.70 14.11 51 

African American 64,655 46.24 14.54 44 

Asian/Pacific Islander 23,143 58.00 13.80 61 

American Indian/Alaska Native 5,011 47.86 14.71 48 

Hispanic 110,390 49.71 14.44 50 

Multiple 12,755 49.33 14.19 49 
Special Instructional Needs 

Economically Disadvantaged 188,470 47.95 14.51 47 

Not-economically Disadvantaged 202,924 52.00 14.03 53 

English Learner (EL) 24,491 46.14 15.77 45 

Non English Learner 366,903 50.31 14.16 50 

Students with Disabilities (SWD) 63,630 45.30 15.35 43 

Students without Disabilities 327,764 50.97 14.05 51 
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Table A.13.3 Summary of SGP Estimates for Subgroups: Grade 6 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 199,403 47.81 14.40 47 

Eamiaté 191,991 52.38 14.12 53 
Ethnicity 

White 174,782 50.70 14.11 51 

African American 64,655 46.24 14.54 44 

Asian/Pacific Islander 23,143 58.00 13.80 61 

American Indian/Alaska Native 5,011 47.86 14.71 48 

Hispanic 110,390 49.71 14.44 50 

Multiple 12,755 49.33 14.19 49 
Special Instructional Needs 

Economically Disadvantaged 188,470 47.95 14.51 47 

Not-economically Disadvantaged 202,924 52.00 14.03 53 

English Learner (EL) 24,491 46.14 15.77 45 

Non English Learner 366,903 50.31 14.16 50 

Students with Disabilities (SWD) 63,630 45.30 15.35 43 

Students without Disabilities 327,764 50.97 14.05 51 
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Table A.13.4 Summary of SGP Estimates for Subgroups: Grade 7 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 195,045 47.17 13.49 46 

Famalé 186,463 52.86 13.39 54 
Ethnicity 

White 175,996 51.07 13.34 52 

African American 61,941 46.67 13.63 45 

Asian/Pacific Islander 23,459 58.08 13.62 61 

American Indian/Alaska Native 3,294 46.54 13.68 45 

Hispanic 105,102 48.48 13.46 48 

Multiple 11,278 48.21 13.42 47 
Special Instructional Needs 

Economically Disadvantaged 173,077 47.75 13.51 47 

Not-economically Disadvantaged 208,431 51.77 13.39 52 

English Learner (EL) 18,886 46.80 14.76 46 

Non English Learner 362,622 50.11 13.38 50 

Students with Disabilities (SWD) 61,503 45.89 14.40 44 

Students without Disabilities 320,005 50.73 13.26 51 
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Table A.13.5 Summary of SGP Estimates for Subgroups: Grade 8 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 191,653 47.82 13.64 47 

Benita 182,755 52.19 13.64 53 
Ethnicity 

White 173,307 49.74 13.52 50 

African American 61,709 46.85 13.75 46 

Asian/Pacific Islander 23,127 57.46 14.11 60 

American Indian/Alaska Native 3,218 51.97 13.77 53 

Hispanic 102,198 50.52 13.66 51 

Multiple 10,474 48.78 13.59 48 
Special Instructional Needs 

Economically Disadvantaged 167,012 48.78 13.67 48 

Not-economically Disadvantaged 207,396 50.89 13.61 51 

English Learner (EL) 18,522 49.68 14.90 50 

Non English Learner 355,886 49.96 13.57 50 

Students with Disabilities (SWD) 59,220 45.95 14.51 44 

Students without Disabilities 315,188 50.70 13.48 51 
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Table A.13.6 Summary of SGP Estimates for Subgroups: Grade 9 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 82,811 48.45 13.43 48 

Famalé 77,682 51.53 13.30 52 
Ethnicity 

White 73,130 50.58 13.13 51 

African American 17,730 47.27 13.47 46 

Asian/Pacific Islander 11,084 58.09 13.66 61 

American Indian/Alaska Native 2,538 51.44 14.10 52 

Hispanic 52,481 48.30 13.58 48 

Multiple 3,223 48.13 13.08 48 
Special Instructional Needs 

Economically Disadvantaged 66,273 48.14 13.56 47 

Not-economically Disadvantaged 94,220 51.22 13.23 52 

English Learner (EL) 9,779 47.00 14.85 46 

Non English Learner 150,714 50.14 13.27 50 

Students with Disabilities (SWD) 25,709 46.12 14.38 45 

Students without Disabilities 134,784 50.68 13.18 51 
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Table A.13.7 Summary of SGP Estimates for Subgroups: Grade 10 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 52,864 48.23 13.04 47 

Eamiaté 50,000 50.95 13.18 51 
Ethnicity 

White 42,767 48.89 12.81 48 

African American 14,127 48.13 13.11 47 

Asian/Pacific Islander 8,646 53.39 13.65 55 

American Indian/Alaska Native 2,163 54.85 13.47 57 

Hispanic 33,721 49.67 13.32 50 

Multiple 1,165 49.68 13.00 51 
Special Instructional Needs 

Economically Disadvantaged 40,998 49.65 13.28 50 

Not-economically Disadvantaged 61,866 49.49 12.99 49 

English Learner (EL) 4,583 46.22 14.56 45 

Non English Learner 98,281 49.71 13.04 50 

Students with Disabilities (SWD) 17,685 46.06 13.66 45 

Students without Disabilities 85,179 50.28 12.99 50 
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Table A.13.8 Summary of SGP Estimates for Subgroups: Grade 11 ELA/L 


Average 
Total Sample Average Standard Median 
Size SGP Error SGP 

Gender 

Male 42,943 47.48 11.95 46 

Eamiaté 39,223 51.85 11.58 53 
Ethnicity 

White 33,122 46.42 11.24 44 

African American 10,385 49.97 12.16 50 

Asian/Pacific Islander 5,907 49.04 11.28 48 

American Indian/Alaska Native 2,058 57.48 12.95 60 

Hispanic 29,501 52.51 12.25 54 

Multiple 921 49.18 11.67 49 
Special Instructional Needs 

Economically Disadvantaged 33,509 52.13 12.31 53 

Not-economically Disadvantaged 48,657 47.80 11.41 47 

English Learner (EL) 3,511 50.96 14.57 52 

Non English Learner 78,655 49.50 11.65 49 

Students with Disabilities (SWD) 14,458 45.84 13.45 44 

Students without Disabilities 67,708 50.36 11.42 51 
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Table A.13.9 Summary of SGP Estimates for Subgroups: Grade 4 Mathematics 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 203,729 50.77 12.87 51 

— 196,386 49.26 12.88 49 
Ethnicity 

White 175,093 52.78 12.61 54 

African American 67,242 44.15 13.24 42 

Asian/Pacific Islander 23,477 58.94 12.86 62 

American Indian/Alaska Native 5,093 43.69 13.24 41 

Hispanic 114,750 47.67 13.06 47 

Multiple 13,613 50.2 12.90 50 
Special Instructional Needs 

Economically Disadvantaged 196,903 46.29 13.10 45 

Not-economically Disadvantaged 203,212 53.65 12.66 55 

English Learner (EL) 35,923 45.63 13.88 44 

Non English Learner 364,192 50.46 12.78 51 

Students with Disabilities (SWD) 62,799 45.94 13.77 44 

Students without Disabilities 337,316 50.79 12.71 51 
Spanish Language Form 2,628 44.32 13.70 42 
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Table A.13.10 Summary of SGP Estimates for Subgroups: Grade 5 Mathematics 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 199,744 49.85 14.42 50 

ee 192,326 50.26 14.53 50 
Ethnicity 

White 174,975 51.07 14.17 52 

African American 64,597 45.68 15.00 44 

Asian/Pacific Islander 23,456 57.85 14.11 61 

American Indian/Alaska Native 4,983 48.61 14.87 48 

Hispanic 110,647 49.43 14.71 49 

Multiple 12,748 49.45 14.52 49 
Special Instructional Needs 

Economically Disadvantaged 188,948 48.09 14.80 47 

Not-economically Disadvantaged 203,122 51.87 14.17 53 

English Learner (EL) 25,304 48.76 15.72 48 

Non English Learner 366,766 50.14 14.39 50 

Students with Disabilities (SWD) 63,447 47.21 15.19 46 

Students without Disabilities 328,623 50.60 14.34 51 
Spanish Language Form 1,774 45.95 15.40 44 
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Table A.13.11 Summary of SGP Estimates for Subgroups: Grade 6 Mathematics 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 195,186 49.03 14.39 49 

Female 187,279 51.06 14.39 51 
Ethnicity 

White 172,709 52.40 13.98 53 

African American 62,017 44.58 15.08 42 

Asian/Pacific Islander 23,278 56.53 13.96 59 

American Indian/Alaska Native 4,502 47.55 14.99 47 

Hispanic 107,713 48.06 14.73 47 

Multiple 11,753 49.73 14.37 49 
Special Instructional Needs 

Economically Disadvantaged 180,485 47.11 14.86 46 

Not-economically Disadvantaged 201,980 52.63 13.97 54 

English Learner (EL) 20,704 44.76 16.15 43 

Non English Learner 361,761 50.32 14.29 50 

Students with Disabilities (SWD) 61,982 45.19 15.47 43 

Students without Disabilities 320,483 50.96 14.18 51 
Spanish Language Form 1,418 46.80 15.91 45 
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Table A.13.12 Summary of SGP Estimates for Subgroups: Grade 7 Mathematics 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 187,774 49.73 15.98 50 

Female 180,096 50.32 15.88 50 
Ethnicity 

White 168,681 51.42 15.31 52 

African American 60,577 47.36 17.03 46 

Asian/Pacific Islander 20,178 56.93 14.66 60 

American Indian/Alaska Native 3,261 42.20 17.27 39 

Hispanic 104,081 48.22 16.51 47 

Multiple 10,645 49.57 15.84 49 
Special Instructional Needs 

Economically Disadvantaged 171,535 47.86 16.73 47 

Not-economically Disadvantaged 196,335 51.90 15.24 53 

English Learner (EL) 19,631 46.43 18.37 45 

Non English Learner 348,239 50.22 15.79 50 

Students with Disabilities (SWD) 60,794 44.74 17.65 42 

Students without Disabilities 307,076 51.06 15.59 51 
Spanish Language Form 1,102 48.44 18.15 46 


March 27, 2018 Page 427 


PARCC 2017 Technical Report 


Table A.13.13 Summary of SGP Estimates for Subgroups: Grade 8 Mathematics 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 149,935 48.71 16.31 48 

ee 138,636 51.27 16.23 52 
Ethnicity 

White 128,198 51.49 15.60 52 

African American 51,210 45.75 17.35 44 

Asian/Pacific Islander 11,803 57.15 14.97 60 

American Indian/Alaska Native 2,731 49.44 17.71 49 

Hispanic 86,506 49.19 16.78 49 

Multiple 7,762 49.23 16.28 49 
Special Instructional Needs 

Economically Disadvantaged 145,380 48.34 16.89 48 

Not-economically Disadvantaged 143,191 51.56 15.64 52 

English Learner (EL) 18,197 48.69 18.27 48 

Non English Learner 270,374 50.02 16.14 50 

Students with Disabilities (SWD) 54,206 46.32 17.84 45 

Students without Disabilities 234,365 50.78 15.91 51 
Spanish Language Form 1,202 53.61 17.66 53 
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Table A.13.14 Summary of SGP Estimates for Subgroups: Algebra | 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 105,870 51.08 15.67 52 

Female 102,340 49.34 15.62 49 
Ethnicity 

White 91,375 52.44 14.46 53 

African American 35,994 47.55 17.43 47 

Asian/Pacific Islander 15,830 56.22 12.98 58 

American Indian/Alaska Native 2,637 45.88 18.75 43 

Hispanic 56,707 46.85 17.08 45 

Multiple 5,373 50.76 15.04 51 
Special Instructional Needs 

Economically Disadvantaged 82,463 46.82 17.28 46 

Not-economically Disadvantaged 125,747 52.46 14.57 54 

English Learner (EL) 9,850 47.59 19.55 47 

Non English Learner 198,360 50.36 15.45 51 

Students with Disabilities (SWD) 30,860 47.94 18.31 47 

Students without Disabilities 177,350 50.62 15.18 51 
Spanish Language Form 1,152 46.41 19.14 45 
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Table A.13.15 Summary of SGP Estimates for Subgroups: Geometry 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 60,636 51.81 14.17 53 

Female 58,538 50.12 14.18 50 
Ethnicity 

White 54,741 53.57 13.16 55 

African American 15,410 43.60 16.02 41 

Asian/Pacific Islander 10,196 57.84 12.26 61 

American Indian/Alaska Native 2,144 55.91 16.01 58 

Hispanic 34,258 47.63 15.48 47 

Multiple 2,176 55.19 12.84 57 
Special Instructional Needs 

Economically Disadvantaged 42,277 46.39 15.74 45 

Not-economically Disadvantaged 76,897 53.51 13.31 55 

English Learner (EL) 4,025 43.64 17.99 41 

Non English Learner 115,149 51.24 14.04 52 

Students with Disabilities (SWD) 17,890 45.82 17.03 44 

Students without Disabilities 101,284 51.89 13.67 53 
Spanish Language Form 1,121 49.21 19.16 49 
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Table A.13.16 Summary of SGP Estimates for Subgroups: Algebra II 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 46,329 50.10 15.93 50 

Female 45,431 48.98 16.04 48 
Ethnicity 

White 39,921 50.02 15.05 50 

African American 11,606 45.98 17.61 44 

Asian/Pacific Islander 9,251 54.57 13.98 57 

American Indian/Alaska Native 1,743 48.01 18.35 47 

Hispanic 27,695 48.73 17.22 48 

Multiple 1,308 50.74 15.10 51 
Special Instructional Needs 

Economically Disadvantaged 32,208 47.97 17.35 47 

Not-economically Disadvantaged 59,552 50.40 15.25 51 

English Learner (EL) 2,533 46.65 19.43 45 

Non English Learner 89,227 49.63 15.89 49 

Students with Disabilities (SWD) 11,966 44.93 18.55 43 

Students without Disabilities 79,794 50.24 15.60 50 
Spanish Language Form 775 40.09 18.12 34 
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Table A.13.17 Summary of SGP Estimates for Subgroups: Integrated Mathematics | 


Average 
Total Average Standard Median 
Sample Size SGP Error SGP 

Gender 

Male 4,707 50.56 16.40 51 

Female 4,068 48.58 16.59 48 
Ethnicity 

White 3,174 50.88 15.24 52 

African American 666 51.18 17.61 51 

Asian/Pacific Islander 394 54.66 15.35 55 

American Indian/Alaska Native 92 47.22 17.80 50 

Hispanic 4,222 47.96 17.34 47 

Multiple 224 51.54 16.55 54 
Special Instructional Needs 

Economically Disadvantaged 4,677 49.54 17.29 50 

Not-economically Disadvantaged 4,098 49.77 15.58 49 

English Learner (EL) 1,746 47.78 18.99 46 

Non English Learner 7,029 50.11 15.87 50 

Students with Disabilities (SWD) 1,111 47.24 19.30 45 

Students without Disabilities 7,664 49.99 16.08 50 
Spanish Language Form 27 59.07 21.39 58 
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Addendum: Statistical Summary of the Fall/Winter Block 2016 Administration 


The addendum presents the results of analyses for the fall/winter block 2016 operational 
administration. These results are reported separately from the spring 2017 results because fall testing 
involved a nonrepresentative subset of students testing only for ELA/L grades 9, 10, and 11, as well as 
Algebra |, Geometry, and Algebra II. Both online and paper test forms were administered for each test. 


To organize the addendum, tables are numbered sequentially according to the section represented by 
the tables. The reader can refer back to the corresponding section in the technical report for related 
information on the topic. For example, the first addendum table for Section 5 is numbered ADD.5.1, the 
second addendum table for Section 5 is numbered ADD.5.2, and so on. 


Addendum 5: Test Taker Characteristics 


Table ADD.5.1 State Participation in ELA/L Fall 2016 Operational Tests, by Grade 


English Language Arts/Literacy 


State Category 


Total Grade 9 Grade 10 Grade 11 
N of Students 29,841 4,891 9,608 15,342 
N of CBT 29,629 4,877 9,587 15,165 
PARCC %of CBT 99.3 99.7 99.8 98.8 
N of PBT 212 14 21 177 
% of PBT 1.7 0.3 0.2 1.2 
N of Students 83 : 7 83 
% of PARCC Data 0.3 - - 0.3 
N of CBT 19 7 - 19 
Pl % of CBT 22.9 - - 22.9 
N of PBT 64 7 : 64 
% of PBT 77.1 : - 77.1 
N of Students 10,539 514 5,457 4,568 
% of PARCC Data 35.3 1.7 18.3 15.3 
N of CBT 10,509 511 5,444 4,554 
mM % of CBT 99.7 99.4 99.8 99.7 
N of PBT 30 3 13 14 
% of PBT 1.1 0.6 0.2 0.3 
N of Students 11,884 4,043 3,871 3,960 
% of PARCC Data 39.8 13.6 13.0 13.3 
N of CBT 11,851 4,043 3,863 3,945 
M % of CBT 99.7 99.8 99.8 99.6 
N of PBT 33 10 8 15 
% of PBT 0.8 0.2 0.2 0.4 
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State Category 


English Language Arts/Literacy 


Total Grade 9 Grade 10 Grade 11 
N of Students 7,334 323 280 6,731 
% of PARCC Data 24.6 1.1 0.9 22.6 
a N of CBT 7,249 322 280 6,647 
% of CBT 98.8 99.7 100 98.8 
N of PBT 85 1 - 84 
% of PBT 1.5 0.3 - 1.2 
N of Students 1 1 : - 
% of PARCC Data 0.0 0.0 - . 
ey N of CBT 1 1 - - 
% of CBT 100 100 - : 
N of PBT : = 2 2 
% of PBT 2 = a 2 
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Table ADD.5.2 State Participation in Mathematics Fall 2016 Operational Tests, by Grade 


Mathematics 


Se eee Total Al GO A2 
Alor Students 29,633 7,159 6,827 15,647 
N of CBT 29,452 7,104 6,794 15,554 
PARCC % of CBT 99.4 99.2 99.5 99.4 
N of PBT 181 55 33 93 
% of PBT 0.6 0.8 0.5 0.6 
N of Students 21 4 1 16 
% of PARCC Data 0.1 0.0 0.0 0.1 
N of CBT oa A 1 16 
P % of CBT 100 100 100 100 
N of PBT S - = - 
% of PBT & a «i - 
Nliof Students 9,488 3,834 758 4,896 
% of PARCC Data 32.0 12.9 2.6 16.5 
N of CBT 9,454 3,819 755 4,880 
me % of CBT 99.6 99.6 99.6 99.7 
N of PBT 34 15 3 16 
% of PBT 0.4 0.4 0.4 0.3 
Nok students 10,471 2,638 3,928 3,905 
% of PARCC Data 35.3 8.9 13.3 13.2 
N of CBT 10,446 2,635 3,916 3,895 
Me % of CBT 99.8 99.9 99.7 99.7 
N of PBT 25 3 12 10 
% of PBT 0.2 0.1 0.3 0.3 
Aro Students 9,570 613 2,127 6,830 
% of PARCC Data 32.3 2.1 7.2 23.0 
N of CBT 9,448 576 2,109 6,763 
mM % of CBT 98.7 94.0 99.2 99.0 
N of PBT 122 37 18 67 
% of PBT 1.3 6.0 0.8 1.0 
N of Students 83 70 13 = 
% of PARCC Data 0.3 0.2 0.0 - 
N of CBT 83 70 13 - 
e % of CBT 100 100 100 - 
N of PBT e 7 2 = 
% of PBT Es 2 2 = 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra Il. 
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Table ADD.5.3 State Participation in Spanish Mathematics Fall 2016 Operational Tests, by Grade 


Mathematics 


ee eed Total Al GO A2 
N of Students 170 41 76 53 
N of CBT 170 41 76 53 
PARCC % of CBT 100 100 100 100 
N of PBT : 7 S - 
% of PBT 2 2 2 - 
N of Students 16 16 - - 
% of PARCC Data 9.4 9.4 - - 
MD N of CBT 16 16 - - 
% of CBT 100 100 - - 
N of PBT 7 2 . - 
% of PBT . S “ - 
N of Students 81 21 50 10 
% of PARCC Data 47.6 12.4 29.4 5.9 
N of CBT 81 21 50 10 
M % of CBT 100 100 100 100 
N of PBT - = . - 
% of PBT = Z e - 
N of Students 73 4 26 43 
% of PARCC Data 42.9 2.4 15.3 25.3 
N of CBT 73 4 26 43 
ml % of CBT 100 100 100 100 
N of PBT - 2 = - 
% of PBT 2 = - 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra Il. 
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Table ADD.5.4 All States Combined: Fall 2016 ELA/L Test Takers by Grade and Gender 


: Gender 
Valid Cases 
Grade Mode Female Male 

N N % N % 

ALL 4,891 2,492 51.0 2,399 49.0 

9 CBT 4,877 2,485 51.0 2,392 49.0 
PBT 14 7 50.0 7 50.0 

ALL 9,608 4,829 50.3 4,779 49.7 

10 CBT 9,587 4,822 50.3 4,765 49.7 
PBT 21 7 33.3 14 66.7 

ALL 15,342 6,817 44.4 8,525 55.6 

11 CBT 15,165 6,736 44.4 8,429 55.6 
PBT 177 81 45.8 96 54.2 


Table ADD.5.5 All States Combined: Fall 2016 Mathematics Test Takers by Grade and Gender 


. Gender 
Grade Mode Maleeses Female Male 
N N % N % 
ALL 7,159 3,524 49.2 3,635 50.8 
Al CBT 7,104 3,507 49.4 3,597 50.6 
PBT 55 17 30.9 38 69.1 
ALL 6,827 3,416 50.0 3,411 50.0 
GO CBT 6,794 3,401 50.1 3,393 49.9 
PBT 33 15 45.5 18 54.5 
ALL 15,647 7,953 50.8 7,694 49.2 
A2 CBT 15,554 7,902 50.8 7,652 49.2 
PBT 93 51 54.8 42 45.2 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra Il. 
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Table ADD.5.6 All States Combined: Fall 2016 Spanish-Language Mathematics Test Takers by Grade 
and Gender 


: Gender 
Grade Mode ees Female Male 

N N % N % 

ALL 41 12 29.3 29 70.7 

Al CBT 41 12 29.3 29 70.7 
PBT - - - - - 

ALL 76 34 44.7 42 55.3 

GO CBT 76 34 44.7 42 55.3 
PBT - - - - - 

ALL 53 29 54.7 24 45.3 

A2 CBT 53 29 54.7 24 45.3 
PBT - - - - - 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra Il. 
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Table ADD.5.7 Demographic Information for Fall 2016 Grade 9 ELA/L, Overall and by State 


Demographic PARCC BI MD NJ NM RI 
Econ Dis (%) 34.9 n/a 34.8 35.5 27.9 n/a 
SWD (%) 18.7 n/a 16.3 19.9 6.5 n/a 
EL (%) eal n/a n/r 14 118 n/a 
Male (%) 49.0 n/a 54.1 488 446 n/a 
Female (%) 51.0 n/a 45.9 51.2 55.4 n/a 
AmInd/ANat (%) n/r n/a n/r n/r n/r n/a 
Asian (%) 5.4 n/a n/r 6.2 n/r n/a 
Black/AA (%) 19.6 n/a 12.8 22.0 n/r n/a 
Hisp/Lat (%) 20.4 n/a 9.1 18.4 63.5 n/a 
Wh/Caus (%) 51.0 n/a 71.0 50.7 223 n/a 
NtvHawaii/Pacific (%) n/r n/a n/r n/r n/r n/a 
Two Or More (%) 2.2 n/a 5.1 1.8 n/r n/a 
Unknown (%) 0.9 n/a n/r n/r 9.0 n/a 


Note: Econ Dis = Economically Disadvantaged; SWD = Student with Disabilities; EL = English learner; 
AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific 
Islander; Two or More = two or more races reported; n/r = not reported; n/a = not applicable. 
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Table ADD.5.8 Demographic Information for Fall 2016 Grade 10 ELA/L, Overall and by State 


Demographic PARCC BI MD NJ NM RI 
Econ Dis (%) 37.9 n/a 36.4 39.0 51.1 n/a 
SWD (%) 15.9 n/a 13.4 20.3 n/r n/a 
EL (%) 6.7 n/a 10.7 1.1 74 nia 
Male (%) 49.7 nia 499 49.6 47.9 nia 
Female (%) 50.3 nfa 501 50.4 52.1 nda 
AmInd/ANat (%) 0.3 n/a 0.4 n/r n/r n/a 
Asian (%) 4.0 n/a 3.2 5.3 n/r n/a 
Black/AA (%) 27.3 n/a 29.2 26.3 n/r n/a 
Hisp/Lat (%) 19.5 nfa 186 17.7 60.0 n/a 
Wh/Caus (%) 46.1 nla 45.4 = 48.7. 23.2 n/a 
NtvHawaii/Pacific (%) n/r n/a n/r n/r n/a n/a 
Two Or More (%) 2.5 n/a 3.2 1.4 n/r n/a 
Unknown (%) 0.3 n/a n/a n/r 7.9 n/a 


Note: Econ Dis = Economically Disadvantaged; SWD = Student with Disabilities; EL = English learner; 
AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific 
Islander; Two or More = two or more races reported; n/r = not reported; n/a = not applicable. 
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Table ADD.5.9 Demographic Information for Fall 2016 Grade 11 ELA/L, Overall and by State 


Demographic PARCC BI MD NJ NM RI 
Econ Dis (%) 47.6 41.0 41.9 36.55 58.2 n/a 
SWD (%) 20.2 n/r 203 194 207 n/a 
EL (%) 73 n/r 2.4 1.0 14.3 n/a 
Male (%) 55.6 47.0 56.9 50.3 57.9 n/a 
Female (%) 44.4 53.0 43.1 49.7 42.1 n/a 
AmInd/ANat (%) 5.8 42.2 n/r n/r 12.3 n/a 
Asian (%) 2.3 n/r 2.1 5.2 0.8 n/a 
Black/AA (%) 20.4 n/r 42.3 26.2 2.5 n/a 
Hisp/Lat (%) 36.4 n/r 11.7 184 641 n/a 
Wh/Caus (%) 31.2 n/r 40.6 481 15.2 n/a 
NtvHawaii/Pacific (%) n/r n/r n/r n/r n/r n/a 
Two Or More (%) 1.9 n/r 2.8 1.6 1.4 n/a 
Unknown (%) a 57.8 n/r n/r 3.6 n/a 


Note: Econ Dis = Economically Disadvantaged; SWD = Student with Disabilities; EL = English learner; 
AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific 
Islander; Two or More = two or more races reported; n/r = not reported; n/a = not applicable. 
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2017 Technical Report 


Demographic PARCC BI MD NJ NM RI 

Econ Dis (%) 37.2 n/a 33.5 43.4 36.4 n/r 
SWD (%) 14.9 n/a 13.1 16.1 20.9 n/r 
EL (%) 5.3 n/a 5.9 2.8 12.7 n/r 
Male (%) 50.8 n/a 51.4 498 524 42.9 
Female (%) 49.2 n/a n/r n/r n/r n/r 
Amind/ANat (%) 0.4 n/a n/r n/r n/r n/r 
Asian (%) 3.8 n/a 3.5 5.2 n/r n/r 
Black/AA (%) 27.3 n/a 31.8 27.0 n/r n/r 
Hisp/Lat (%) 20.6 n/a 12.6 23.6 59.9 n/r 
Wh/Caus (%) 43.7 n/a 48.7 41.3 17.6 88.6 
NtvHawaii/Pacific (%) n/r n/a n/r n/r n/r n/r 
Two Or More (%) 2.4 n/a 2.1 2.0 n/r n/r 
Unknown (%) 1.6 n/a n/r n/r 16.8) n/r 


Note: Econ Dis = Economically Disadvantaged; SWD = Student with Disabilities; EL = English learner; 


AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 


Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific 


Islander; Two or More = two or more races reported; n/r = not reported; n/a = not applicable. 
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Table ADD.5.11 Demographic Information for Fall 2016 Geometry, Overall and by State 


Demographic PARCC BI MD NJ NM RI 
Econ Dis (%) 42.7 n/a 33.9 37.4 55.3 n/a 
SWD (%) 17.2 n/a 18.7 17.7 15.9 n/a 
EL (%) 5.3 nfa 3.3 3 11:7 n/a 
Male (%) 50.0 n/a 54.0 49.4 49.7 n/a 
Female (%) 50.0 n/a 46.0 50.6 50.3 n/a 
AmInd/ANat (%) 2.9 n/a n/r n/r 8.7 n/a 
Asian (%) 3.8 n/a n/r Se n/r n/a 
Black/AA (%) 16.4 n/a 20.2 230 27 n/a 
Hisp/Lat (%) 33.1 n/a 135 204 °& 63.6 n/a 
Wh/Caus (%) 40.2 n/a 60.2 48.5 17.8 n/a 
NtvHawaii/Pacific (%) n/r n/a n/a n/r n/r n/a 
Two Or More (%) a2 n/a 3.6 1.7 1.7 n/a 
Unknown (%) 1.6 n/a n/a n/r 4.9 n/a 


Note: Econ Dis = Economically Disadvantaged; SWD = Student with Disabilities; EL = English learner; 
AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific 
Islander; Two or More = two or more races reported; n/r = not reported; n/a = not applicable. 
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Table ADD.5.12 Demographic Information for Fall 2016 Algebra II, Overall and by State 


Demographic PARCC BI MD NJ NM RI 
Econ Dis (%) 45.1 n/a 35.5 35.0 57.6 n/a 
SWD (%) 12.2 n/a 12.1 14.1 11.2 n/a 
EL (%) 5.7 n/a 2.0 18 105 n/a 
Male (%) 49.2 n/a 492 50.6 484 n/a 
Female (%) 50.8 n/a 50.8 49.4 51.6 n/a 
AmInd/ANat (%) 6.6 n/a n/r n/r 14.6 n/a 
Asian (%) 3.3 n/a 3.7 7.0 0.8 n/a 
Black/AA (%) 17.4 n/a 34.8 22.9 1.8 n/a 
Hisp/Lat (%) 35.8 n/a 13.6 18.1 51.8 n/a 
Wh/Caus (%) 33.7 n/a 438 49.9 173 ~ ofa 
NtvHawaii/Pacific (%) n/r n/a n/r n/r n/r n/a 
Two Or More (%) 2:1 n/a 3.8 1.5 141 n/a 
Unknown (%) ae n/a n/a n/r 2.4 n/a 


Note: Econ Dis = Economically Disadvantaged; SWD = Student with Disabilities; EL = English learner; 
AmInd/ANat = American Indian/Alaska Native; Black/AA = Black/African American; Hisp/Lat = 
Hispanic/Latino; Wh/Caus = White/Caucasian; NtvHawaii/Pacific = Native Hawaiian or Other Pacific 
Islander; Two or More = two or more races reported; n/r = not reported; n/a = not applicable. 
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Addendum 8: Reliability 


Table ADD.8.1 shows the total group level reliability estimates and raw score SEM for the fall 2016 forms. Tables ADD.8.2 — ADD.8.9 show the 
subgroup reliability estimates and raw score SEM. A minimum sample size of 100 per core form was required for calculating the reliability 
estimates for subgroups; therefore, the subgroup totals may not equal the total group sample size. Tables ADD.8.10 — ADD.8.12 provide the 
claim and subclaim reliability and raw score SEM estimates for the fall 2016 forms. The paper-based tests did not have sufficient sample sizes for 


reliability analyses. 


Table ADD.8.1 Summary of Test Reliability Estimates for Fall 2016 Total Group 


- Avg. : 

Grade/ Number Maximum Raw Avg. in. Min. Mix: Max. 

Content Mode Possible (ene Sample eegay Sample pian 
Course of forms Score Reliability . Reliability : Reliability 

Score Size Size 
SEM 

ELA/L 9 CBT 2 121 5.87 0.93 166 0.86 4,651 0.94 

ELA/L 10 CBT 2 121 6.01 0.94 156 0.87 9,290 0.94 

ELA/L 11 CBT 2 121 5.74 0.91 151 0.88 8,812 0.92 

Mathematics Al CBT 2 81 4.00 0.92 1,312 0.89 5,105 0.92 

Mathematics GO CBT 2 81 3.44 0.92 623 0.86 3,942 0.93 

Mathematics A2 CBT 2 81 3.46 0.94 610 0.91 7,655 0.94 


Note: Al = Algebra |, GO = Geometry, A2 = Algebra II. ELA/L grades 9 and 10, Algebra |, and Algebra II had insufficient sample sizes for PBT. 
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Table ADD.8.2 Summary of Test Reliability Estimates for Fall 2016 Subgroups: Grade 9 ELA/L 


CBT 
Max. Avg. Min. Min. Max. Max. 
Raw Avg.  Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility 
Gender 
Male 121 5.66 0.93 104 0.85 2,252 0.94 
eainala 121 610 0.93 2,399 0.93 2,399 0.93 
Ethnicity 
White 121 6.02 0.94 2,404 0.94 2,404 0.94 
Atricanamerican 121 5.76 0.91 874 0.91 874 0.91 
Asian/Pacific Islander 121 622 0.92 273 —s-:0.92 273 0.92 
Am. Indian/Alaska Native 
Hispanic 121 5.70 0.92 946 ~=— 0.92 946 0.92 
Multiple 121 5.93 0.94 104s «0.94 104 0.94 
Special Instructional Needs 
Economiteyy 121 5.79 0.92 1,570 0.92 1,570 0.92 
Disadvantaged 
NOE Economically 121 600 094 2,943 094 2,943 0.94 
Disadvantaged 
English Learner 
Non-English Learner 121 5.90 0.93 164 0.86 4,489 0.94 
Siidentswith Disabilities 121 5.06 0.90 166 0.86 733 0.91 
Students w/o Disabilities 121 6.05 0.93 3,785 0.93 3,785 0.93 
Students Taking 
Accommodated Forms 
A: ASL 
C: Closed Caption 
R: Screen Reader 
T: Text-to-Speech 121 4.58 0.86 165 0.86 165 0.86 
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Table ADD.8.3 Summary of Test Reliability Estimates for Fall 2016 Subgroups: Grade 10 ELA/L 


CBT 
Max. Avg. Min. Min. Max. Max. 
Raw Avg.  Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility 
Gender 
Male 121 5.77 0.94 106 O86 4,591 0.94 
eainala 121 6.25 0.94 4699 0.94 4,699 0.94 
Ethnicity 
White 121 6.30 0.93 4,343 0.93 4,343 0.93 
African American 121 5.58 0.92 2,476 0.92 2,476 0.92 
Asian/Pacific Islander 121 630 0.94 385 ~—s-: 0.94 385 0.94 
Am. Indian/Alaska Native 
Hispanic 121 5.76 0.93 1,808 0.93 1,808 0.93 
Multiple 121 6.30 0.93 235 0.93 235 0.93 
Special Instructional Needs 
Economitauy 121 5.78 0.92 3483 0.92 3,483 0.92 
Disadvantaged 
Not Economically 121 6.16 0.94 5,715 0.94 5,715 0.94 
Disadvantaged 
English Learner 121 449 0.76 629 ~—s0.76 629 0.76 
Non-English Learner 121 609 0.94 152. 0.87 + 8,611 0.94 
Students with Disabilities 121 5.46 0.93 156 0.87 1,345 0.93 
Students w/o Disabilities 121 6.10 0.94 7,824 0.94 7,824 0.94 
Students Taking 
Accommodated Forms 
A: ASL 
C: Closed Caption 
R: Screen Reader 
T: Text-to-Speech 121 4.55 0.84 151 0.84 151 0.84 
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Table ADD.8.4 Summary of Test Reliability Estimates for Fall 2016 Subgroups: Grade 11 ELA/L 


CBT 
Max. Avg. Min. Min. Max. Max. 
Raw Avg.  Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility 
Gender 
Male 121 5.45 0.91 4566 0.91 4,566 0.91 
eainala 121 6.07 0.91 4246 0.91 4,246 0.91 
Ethnicity 
White 121 5.97 0.92 3,587 0.92 3,587 0.92 
Atricanamerican 121 5.47 0.88 2712 “O88 2,712 0.88 
Asian/Pacific Islander 121 625 0.93 317. —s:0.93 317 0.93 
Am. Indian/Alaska Native 121 5.55 0.89 148 ~=— 0.89 148 0.89 
Hispanic 121 5.68 0.91 1,825 0.91 1,825 0.91 
Multiple 121 5.97 0.92 3,587 0.92 3,587 0.92 
Special Instructional Needs 
ee 121 578 0.90 179 090 179 0.90 


Disadvantaged 
Not Economically 


: 121 5.58 0.89 3,360 0.89 3,360 0.89 
Disadvantaged 


English Learner 121 5.87 0.92 5,365 0.92 5,365 0.92 
Non-English Learner 121 462 0.80 247 080 #247 0.80 
Students with Disabilities 121 5.77. 0.91 146 0.89 = 8,525 0.91 
Students w/o Disabilities 121 5.04 0.87 1,513 0.87 151 0.88 


Students Taking 
Accommodated Forms 


A: ASL 
C: Closed Caption 


R: Screen Reader 
T: Text-to-Speech 121 4.49 0.88 146 0.88 146 0.88 
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Table ADD.8.5 Summary of Test Reliability Estimates for Fall 2016 Subgroups: Algebra | 


CBT 
Max. Avg. Min. Min. Max. Max. 
Raw Avg.  Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility 
Gender 
Male 81 3.93 0.92 691 0.90 2,504 0.93 
eainala 81 4.07 0.91 621 0.89 2,601 0.92 
Ethnicity 
White 81 4.14 0.91 880 0.88 2,086 0.92 
African American 81 3.51 0.89 173 0.86 1,525 0.89 
Asian/Pacific Islander 81 419 0.91 223 0.91 223 0.91 
Am. Indian/Alaska Native 
Hispanic 81 3.77 0.89 179 0.87 1,078 0.90 
Multiple 81 4.18 0.93 135 0.93 135 0.93 
Special Instructional Needs 
Economitauy 81 3.74 0.89 366 0.88 1,941 0.89 
Disadvantaged 
NOE Economically 81 4.09 0.92 946 0.89 3,089 0.93 
Disadvantaged 
English Learner 81 2.78 0.70 230 0.70 230 0.70 
Non-English Learner 81 4.02 0.92 1,229 0.89 4,827 0.92 
Students with Disabilities 81 3.45 0.89 253 0.89 581 0.89 
Students w/o Disabilities 81 4.04 0.92 1,058 0.89 4,442 0.92 
Students Taking 
Accommodated Forms 
A: ASL 
C: Closed Caption 
R: Screen Reader 
T: Text-to-Speech 81 4.06 0.89 1,285 0.89 1,285 0.89 
Students Taking Translated 
Forms 
Spanish-Language Form 
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Table ADD.8.6 Summary of Test Reliability Estimates for Fall 2016 Subgroups: Geometry 


CBT 
Max. Avg. Min. Min. Max. Max. 
Raw Avg. Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility 
Gender 
Male 81 3.39 0.92 327. -0.87.—s:1,979 0.93 
eainala 81 3.48 0.91 296 0.85 ~=«-:1,963 0.92 
Ethnicity 
White 81 3.61 0.91 366 0.84 1,922 0.92 
Atricanamerican 81 3.06 0.85 100 =: 0.77 769 0.86 
Asian/Pacific Islander 81 3.98 0.92 229 0.92 229 0.92 
Am. Indian/Alaska Native 
Hispanic 81 3.12 0.88 135 0.74 906 0.90 
Multiple 
Special Instructional Needs 
Economitauy 81 3.16 0.87 213 «0.78 «=1,429.—Ss0.89 
Disadvantaged 
NOE Economically 81 3.57 0.92 410 086 2,419 0.93 
Disadvantaged 
English Learner 
Non-English Learner 81 3.45 0.92 561 0.86 3,809 0.92 
Students with Disabilities 81 2.83 0.84 202 0.72 593 0.87 
Students w/o Disabilities 81 3.54 0.92 420 0.86 3,190 0.92 
Students Taking 
Accommodated Forms 
A: ASL 
C: Closed Caption 
R: Screen Reader 
T: Text-to-Speech 81 3.19 0.86 610 0.86 610 0.86 
Students Taking Translated 
Forms 
Spanish-Language Form 
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Table ADD.8.7 Summary of Test Reliability Estimates for Fall 2016 Subgroups: Algebra II 


CBT 
Max. Avg. Min. Min. Max. Max. 
Raw Avg.  Relia- Sample Relia- Sample Relia- 
Score SEM bility Size bility Size bility 
Gender 
Male 81 3.43 0.94 318 0.91 3,744 0.95 
eainala 81 3.49 0.93 292 0.91 ~—s« 3,911 0.93 
Ethnicity 
White 81 3.79 0.93 347 0.90 3,198 0.94 
African American 81 2.87 0.86 2,061 0.86 2,061 0.86 
Asian/Pacific Islander 81 4.02 0.94 437. —s «0.94 437 0.94 
Am. Indian/Alaska Native 81 2.98 0.87 101 0.87 101 0.87 
Hispanic 81 3.11 0.89 115 0.85 1627 0.89 
Multiple 81 3.56 0.94 191 0.94 191 0.94 
Special Instructional Needs 
Economitauy 81 3.06 0.89 205 «0.87 «2,642.~—Ss0.89 
Disadvantaged 
NOE Economically 81 3.64 094 405 091 4931 0.94 
Disadvantaged 
English Learner 81 2.58 0.75 1740.75 174 0.75 
Non-English Learner 81 3.48 0.94 534 -0.91_~—s_ 7,452 0.94 
Students with Disabilities 81 2.91 0.91 171 0.85 857 0.92 
Students w/o Disabilities 81 3.52 0.94 439 0.91 6,674 0.94 
Students Taking 
Accommodated Forms 
A: ASL 
C: Closed Caption 
R: Screen Reader 
T: Text-to-Speech 81 3.47 0.91 599 0.91 599 0.91 
Students Taking Translated 
Forms 
Spanish-Language Form 
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Table ADD.8.8 Fall 2016 Average ELA/L Reliability Estimates for Reading Total and Subscores 


Reading: Reading: Reading: Reading: 
Total Literature Information Vocabulary 
Grade Max Possible Max Possible Max Possible Max Possible 
Mod Reliabilit Reliabilit Reliabilit Reliabilit 

Level is Raw Score ellability Raw Score eHability Raw Score ellapitity Raw Score elability 

9 CBT 76 0.90 20 0.71 40 0.84 16 0.63 

10 CBT 76 0.91 32 0.77 30 0.84 14 0.63 

11 CBT 76 0.86 28 0.70 32 0.73 16 0.55 

Table ADD.8.9 Fall 2016 Average ELA/L Reliability Estimates for Writing Total and Subscores 
nN: WHI epee rss e and 
Total Written Expression . 3 ‘7 
Conventions 
d Max Possible R us Max Possible R wags Max Possible R etuate 

Grade Mod ax Possible Raw Reliability ax Possible Raw Reliability ax Possible Raw Reliability 
Level Score Score Score 

9 CBT 4S 0.88 36 0.88 9 0.89 

10 CBT 45 0.88 36 0.89 9 0.90 

11 CBT 45 0.87 36 0.87 9 0.87 
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Table ADD.8.10 Fall 2016 Average Mathematics Reliability Estimates for Total Test and Subscores 


Major Content 


Additional & Mathematics 


Modeling Practice 


Supporting Content Reasoning 
Max Max Max Max 
Grade Level Mode Possible: -pciiabiity OO’ “Ranabitigy, OP’ Retaptigy. "= Reliability 

Raw Raw Raw Raw 

Score Score Score Score 
Al CBT 28 0.81 21 0.74 14 0.65 18 0.64 
GO CBT 30 0.83 19 0.66 14 0.56 18 0.68 
A2 CBT 27 0.80 22 0.76 14 0.79 18 0.72 


Note: A1 = Algebra |, GO = Geometry, A2 = Algebra Il. 
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Tables ADD.8.11 and ADD.8.12 provide information about the accuracy and the consistency of two 
classifications made on the basis of the scores on the fall block 2016 English language arts/literacy and 
mathematics assessments respectively. The columns labeled “Exact level” provide the classification of 
the student into one of five achievement levels. The columns labeled “Level 4 or higher vs. 3 or lower” 
provide the classification of the student as being either in one of the upper two levels (Levels 4 and 5) or 
in one of the lower three levels (Levels 1, 2, and 3). 


Tables ADD.8.13 to ADD.8.18 provide more detailed information about the accuracy and the consistency 
of the classification of students into proficiency levels for each fall block 2016 PARCC assessment. Each 
cell in the 5-by-5 table shows the estimated proportion of students who would be classified into a 
particular combination of proficiency levels. The sum of the five bold values on the diagonal should 
equal the exact level of decision accuracy or consistency presented in Tables ADD.8.110r ADD.8.12 for 
the corresponding PARCC assessment. For “Level 4 and higher vs. 3 and lower” found in Tables 
ADD.8.11 or ADD.8.12, the sum of the shaded values in Tables ADD.8.13 to ADD.8.18 should equal the 
level of decision accuracy or consistency for the corresponding PARCC assessment in ADD.8.11 or 
ADD.8.12. Note that the sums based on values may not match exactly to the values due to truncation 
and rounding. 
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Table ADD.8.11 Reliability of Classification: Summary for ELA/L Fall Block 2016 


Decision Accuracy: Decision Consistency: 

Proportion Accurately Proportion Consistently 

Classified Classified 
Level 4 or Level 4 or 
Grade Testing higher vs. 3 or higher vs. 3 or 

Level Mode Exact level lower Exact level lower 
9 CBT 0.76 0.92 0.67 0.89 
10 CBT 0.75 0.93 0.66 0.90 
11 CBT 0.74 0.94 0.64 0.91 


Table ADD.8.12 Reliability of Classification: Summary for Mathematics Fall Block 2016 


Decision Accuracy: Decision Consistency: 
Proportion Accurately Proportion Consistently 
Classified Classified 
Level 4 or Level 4 or 
Grade Testing higher vs. 3 or higher vs. 3 or 
Level Mode Exact Level lower Exact Level lower 
re CBT 0.77 0.92 0.68 0.88 
PBT -- -- -- -- 
CBT 0.78 0.95 0.70 0.92 
GO 
PBT -- -- -- -- 
CBT 0.82 0.96 0.75 0.95 
A2 
PBT = -- 2a = 
Note: Ail = Algebra |, GO = Geometry, A2 = Algebra Il. “--” means insufficient sample size (< 100 
students). 
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Full 
Summative Level1 Level2  Level3 Level 4 Level 5 carey 
Scale Score or. 
650 — 699 0.09 0.01 0.00 0.00 0.00 0.11 
Decision 700 — 724 0.02 0.11 0.03 0.00 0.00 0.17 
Accuracy 725 — 749 0.00 0.03 0.18 0.04 0.00 0.25 
750 — 809 0.00 0.00 0.04 0.29 0.03 0.36 
810 -— 850 0.00 0.00 0.00 0.02 0.08 0.10 
ea 650 — 699 0.09 0.02 0.00 0.00 0.00 0.12 
Decision 700 — 724 0.03 0.10 0.05 0.00 0.00 0.17 
Consistency 725 — 749 0.00 0.04 0.15 0.06 0.00 0.24 
750 — 809 0.00 0.00 0.06 0.26 0.03 0.35 
810 -— 850 0.00 0.00 0.00 0.03 0.08 0.11 
Table ADD.8.14 Reliability of Classification: Grade 10 ELA/L 
Full 
Summative Level1 Level2  Level3 Level 4 Level 5 cateeony 
Scale Score _ 
650 — 699 0.23 0.02 0.00 0.00 0.00 0.25 
Decision 700 — 724 0.03 0.09 0.04 0.00 0.00 0.16 
Accuracy 725 — 749 0.00 0.03 0.11 0.04 0.00 0.18 
750 — 809 0.00 0.00 0.04 0.22 0.02 0.28 
810 -— 850 0.00 0.00 0.00 0.02 0.10 0.12 
oad 650 — 699 0.22 0.03 0.01 0.00 0.00 0.26 
Decision 700 — 724 0.04 0.07 0.05 0.01 0.00 0.16 
Consistency 725-749 0.00 0.04 0.09 0.05 0.00 0.18 
750 — 809 0.00 0.01 0.05 0.19 0.03 0.27 
810 -— 850 0.00 0.00 0.00 0.04 0.10 0.13 
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Table ADD.8.15 Reliability of Classification: Grade 11 ELA/L 


Full 


Summative Level1 Level2  Level3 Level 4 Level 5 eo 
Scale Score note 
650 — 699 0.26 0.04 0.00 0.00 0.00 0.30 
Decision 700 — 724 0.04 0.19 0.05 0.00 0.00 0.29 
Accuracy 725 — 749 0.00 0.05 0.14 0.03 0.00 0.23 
750 — 809 0.00 0.00 0.03 0.12 0.01 0.16 
810-850 0.00 0.00 0.00 0.00 0.02 0.02 
ea 650 — 699 0.24 0.06 0.00 0.00 0.00 0.31 
Decision 700 — 724 0.05 0.15 0.06 0.00 0.00 0.27 
Consistency 725 — 749 0.00 0.06 0.11 0.04 0.00 0.22 
750 — 809 0.00 0.00 0.04 0.11 0.01 0.17 
810 -— 850 0.00 0.00 0.00 0.01 0.02 0.03 
Table ADD.8.16 Reliability of Classification: Algebra | 
Full 
Summative Level1 Level2  Level3 Level 4 Level 5 cateeony 
Scale Score _ 
650 — 699 0.14 0.03 0.00 0.00 0.00 0.16 
Decision 700 — 724 0.03 0.16 0.03 0.00 0.00 0.22 
Accuracy 725 —749 0.00 0.05 0.14 0.05 0.00 0.23 
750 — 809 0.00 0.00 0.03 0.32 0.01 0.37 
810 -— 850 0.00 0.00 0.00 0.01 0.01 0.02 
i 650 — 699 0.13 0.04 0.00 0.00 0.00 0.18 
Decision 700 — 724 0.04 0.13 0.04 0.00 0.00 0.21 
Consistency 725-749 0.00 0.06 0.11 0.06 0.00 0.23 
750 — 809 0.00 0.00 0.05 0.30 0.01 0.36 
810 -— 850 0.00 0.00 0.00 0.01 0.01 0.03 
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Table ADD.8.17 Reliability of Classification: Geometry 


Full 


Summative Level1 Level2  Level3 Level 4 Level 5 cateeony 
Scale Score Lae 
650 — 699 0.12 0.03 0.00 0.00 0.00 0.15 
Decision 700 — 724 0.03 0.31 0.04 0.00 0.00 0.38 
Accuracy 725 — 749 0.00 0.05 0.22 0.03 0.00 0.30 
750 — 809 0.00 0.00 0.02 0.14 0.01 0.16 
810 -— 850 0.00 0.00 0.00 0.00 0.01 0.01 
_ 650 — 699 0.11 0.05 0.00 0.00 0.00 0.16 
Decision 700 — 724 0.04 0.26 0.06 0.00 0.00 0.36 
Consistency 725 — 749 0.00 0.07 0.19 0.04 0.00 0.29 
750 — 809 0.00 0.00 0.04 0.13 0.01 0.17 
810 — 850 0.00 0.00 0.00 0.01 0.01 0.01 
Table ADD.8.18 Reliability of Classification: Algebra II 
Full 
Summative Level1 Level2  Level3 Level 4 Level 5 rae 
Scale Score none 
650 — 699 0.45 0.04 0.00 0.00 0.00 0.49 
Decision 700 — 724 0.04 0.18 0.03 0.00 0.00 0.25 
Accuracy 725 —749 0.00 0.03 0.09 0.02 0.00 0.14 
750 — 809 0.00 0.00 0.01 0.10 0.00 0.12 
810 -— 850 0.00 0.00 0.00 0.00 0.00 0.01 
650 — 699 0.43 0.06 0.00 0.00 0.00 0.50 
Decision 700 — 724 0.05 0.15 0.03 0.00 0.00 0.24 
Consistency 725 — 749 0.00 0.04 0.07 0.02 0.00 0.14 
750 — 809 0.00 0.00 0.02 0.09 0.00 0.12 
810 -— 850 0.00 0.00 0.00 0.00 0.00 0.01 
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Addendum 9: Validity 


The intercorrelations for the fall 2016 tests are presented in Tables ADD.9.1 through ADD.9.3 for ELA/L 
grades 9, 10, and 11 and Tables ADD.9.4 though ADD.9.6 for the traditional mathematics courses (A1, 
GO, A2). Like the spring intercorrelations, the ELA/L all have moderate to high values with the writing 
subclaims being highly intercorrelated. The mathematics intercorrelations have moderate values. Tables 
ADD.9.7 through ADD.9.9 are the correlations between ELA/L and mathematics from the fall block. 


Table ADD.9.1 Average Intercorrelations and Reliability between Grade 9 ELA/L Subclaims 


CBT 

RD RL RI RV. WR WE WKL 
RD 0.90 4,855 4,855 4,855 4,855 4,855 4,855 
RL 0.87 0.71 4,855 4,855 4,855 4,855 4,855 
RI 0.96 0.76 0.84 4,855 4,855 4,855 4,855 
RV 0.84 0.65 0.72 0.63 4,855 4,855 4,855 
WR 0.75 0.71 0.73 0.55 0.88 4,855 4,855 
WE 0.75 0.71 0.73 0.55 1.00 0.88 4,855 
WKL 0.75 0.70 0.73 0.55 0.98 0.97 0.89 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = 
Writing, WE = Written Expression, and WKL = Writing Knowledge and Conventions. The shaded values 
along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are 
provided in the lower portion of the table and the total sample sizes are provided in the upper portion 
of the table. 
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Table ADD.9.2 Average Intercorrelations and Reliability between Grade 10 ELA/L Subclaims 


CBT 

RD RL RI RV WR WE WKL 
RD 0.91 9,533 9,533 9,533 9,533 9,53 9,533 
RL 0.93 0.77 9,533 9,533 9,533 9,533 9,533 
RI 0.94 0.80 0.84 9,533 9,533 9,533 9,533 
RV 0.82 0.67 0.71 0.63 9,533 9,533 9,533 
WR 0.80 0.75 0.77 0.60 0.88 9,533 9,533 
WE 0.79 0.75 0.77 0.60 1.00 0.89 9,533 
WKL 0.79 0.74 0.77 0.61 0.98 0.97 0.90 

Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = 


Writing, WE = Written Expression, and WKL = Writing Knowledge and Conventions. The shaded values 
along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are 
provided in the lower portion of the table and the total sample sizes are provided in the upper portion 
of the table. 


Table ADD.9.3 Average Intercorrelations and Reliability between Grade 11 ELA/L Subclaims 


CBT 

RD RL RI RV. WR. WE WKL 
RD 0.86 9,230 9,230 9,230 9,230 9,230 9,230 
RL 0.91 0.70 9,230 9,230 9,230 9,230 9,230 
RI 0.91 0.71 0.73 9.230 9,230 9,230 9,230 
RV 0.80 0.50 0.52 0.55 9,230 9,230 9,230 
WR 0.76 0.70 0.74 0.52 0.87 9,230 9,230 
WE 0.76 0.70 0.73 0.52 1.00 0.87 9,230 
WKL 0.75 0.69 0.73 0.52 0.98 0.96 0.87 


Note: RD = Reading, RL = Reading Literature, RI = Reading Information, RV = Reading Vocabulary, WR = 
Writing, WE = Written Expression, and WKL = Writing Knowledge and Conventions. The shaded values 
along the diagonal are the reliabilities as reported in Section 8. The average intercorrelations are 
provided in the lower portion of the table and the total sample sizes are provided in the upper portion 
of the table. 


March 27, 2018 Page 460 


PARCC 2017 Technical Report 


Table ADD.9.4 Average Intercorrelations and Reliability between Algebra | Subclaims 


CBT 
MC ASC MR MP 
MC 081 6,892 6,892 6,892 
ASC 0.80 0.74 6,892 6,892 
MR 0.74 0.72 0.65 6,892 
MP 0.79 0.75 0.71 0.64 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, 
and MP = Modeling Practice. The shaded values along the diagonal are the reliabilities as reported in 
Section 8. The average intercorrelations are provided in the lower portion of the table and the total 
sample sizes are provided in the upper portion of the table. 


Table ADD.9.5 Average Intercorrelations and Reliability between Geometry Subclaims 


CBT 
MC ASC MR MP 
MC 0.83 5,148 5,148 5,148 
ASC 0.74 066 5,148 5,148 
MR 0.71 062 056 5,148 
MP 0.78 068 0.70 0.68 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, 
and MP = Modeling Practice. The shaded values along the diagonal are the reliabilities as reported in 
Section 8. The average intercorrelations are provided in the lower portion of the table and the total 
sample sizes are provided in the upper portion of the table. 
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Table ADD.9.6 Average Intercorrelations and Reliability between Algebra II Subclaims 


CBT 
MC ASC MR MP 
MC 0.80 8,958 8,958 8,958 
ASC 0.78 0.76 8,958 8,958 
MR 0.78 0.79 0.79 8,958 
MP 0.79 0.76 0.77 0.72 


Note: MC = Major Content, ASC = Additional and Supporting Content, MR = Mathematical Reasoning, 
and MP = Modeling Practice. The shaded values along the diagonal are the reliabilities as reported in 
Section 8. The average intercorrelations are provided in the lower portion of the table and the total 
sample sizes are provided in the upper portion of the table. 


Table ADD.9.7 Average Correlations between ELA/L and Mathematics for High School 
CBT 


Al GO A2 
9 0.75 0.73 
(1,052) (459) 
10 0.67 0.68 0.80 
(714) (1,032) (963) 
11 0.42 0.47 0.52 
(292) (1,528) (5,713) 
Note: ELA/L = English language arts/literacy, Al = Algebra 1, GO = Geometry, A2 = Algebra Il. The 
correlations are provided with the sample sizes, below in parentheses. 


ELA/L 
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Table ADD.9.8 Average Correlations between Reading and Mathematics for High School 
CBT 


Al GO A2 
9 0.73 0.71 
(1,052) (459) 
10 0.67 0.66 0.80 
(714) (1,032) (963) 
11 0.45 0.46 0.52 
(292) (1,528) (5,713) 
Note: RD = Reading, A1 = Algebra |, GO = Geometry, A2 = Algebra II. The correlations are provided with 
the sample sizes, below in parentheses. 


RD 


Table ADD.9.9 Average Correlations between Writing and Mathematics for High School 
CBT 


Al GO A2 
9 0.68 0.64 
(1,052) (459 
10 0.55 0.58 0.70 
(714) (1,032) (963) 
11 0.26 0.35 0.41 
(292) (1,528) (5,713) 
Note: WR = Writing, Al = Algebra |, GO = Geometry, A2 = Algebra Il. The average correlations are 
provided with the sample sizes, below in parentheses. 


WR 
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Table ADD.12.1 Fall 2016 Subgroup Performance for ELA/L Scale Scores: Grade 9 


Group Type Group 
Full Summative Score 
Gender Female 
Male 
Ethnicity American Indian/Alaska Native 


Economic Status” 


English Learner 
Status 


Disabilities 


Reading Score 
Gender 


Ethnicity 


Economic Status” 


English Learner 
Status 


Disabilities 


Writing Score 
Gender 
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Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 

Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 


Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 


N 
4,891 
2,492 
2,399 

n/r 
265 
960 

1,000 
n/r 
2,493 
107 
1,709 
3,040 
101 


4,718 
914 
3,840 
4,891 
2,492 
2,399 
n/r 
265 
960 
1,000 
n/r 
2,493 
107 
1,709 
3,040 


101 
4,718 

914 
3,840 
4,891 
2,492 


Mean 
745.15 
752.37 
737.65 

n/r 
767.18 
731.12 
733.54 

n/r 
753.18 
739.84 
732.77 
752.59 
699.38 


746.05 
715.68 
752.06 
47.27 
49.13 
45.34 
51.25 
55.35 
42.06 
42.52 
n/r 
50.40 
45.46 
42.49 
50.15 


30.68 
47.61 
36.59 
49.78 
31.89 
35.08 


SD 
36.10 
34.88 
35.83 

n/r 
30.93 
32.22 
33.30 

n/r 
35.40 
36.92 
33.90 
35.26 
26.64 


35.62 
30.53 
33.85 
14.20 
13.77 
14.39 
20.20 
12.53 
12.47 
12.77 

n/r 
14.11 
14.91 
13.04 
14.04 


10.25 
14.05 
12.15 
13.49 
12.54 
11.32 


Min 
650 
650 
650 

n/r 
669 
650 
650 

n/r 
650 
650 
650 
650 
655 


650 
650 
650 
10 
10 
10 
19 
19 
10 
10 
n/r 
10 
12 
10 
10 


13 
10 
10 
10 
10 
10 


Max 
850 
850 
843 

n/r 
840 
837 
843 

n/r 
850 
830 
840 
850 
796 


850 
826 
850 
90 
90 
90 
78 
88 
90 
90 
n/r 
90 
85 
90 
90 


62 
90 
88 
90 
60 
60 
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Group Type 


Ethnicity 


Economic Status® 


English Learner 
Status 


Disabilities 


Group 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 


Students without Disabilities 


N 
2,399 
n/r 
265 
960 
1,000 
n/r 
2,493 
107 
1,709 
3,040 


101 
4,718 

914 
3,840 


Mean 
28.57 
32.50 
38.57 
27.56 
28.92 

n/r 
34.16 
30.27 
28.16 
34.11 


16.01 
32.19 
22.07 
34.18 


2017 Technical Report 


SD 
12.89 
14.95 

9.50 
12.62 
12.63 

n/r 
11.90 
12.63 
12.95 
11.77 


10.21 
12.38 
12.28 
11.48 


10 


60 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP): receipt of 
free or reduced-price lunch (FRL). n/r = not reported due to n<25. 
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Table ADD.12.2 Fall 2016 Subgroup Performance for ELA/L Scale Scores: Grade 10 


Group Type Group 
Full Summative Score 
Gender Female 
Male 
Ethnicity American Indian/Alaska Native 


Economic Status" 


English Learner 
Status 


Disabilities 


Reading Score 
Gender 


Ethnicity 


Economic Status" 


English Learner 
Status 


Disabilities 


Writing Score 
Gender 
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Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 
Male 


N 
9,608 
4,829 
4,779 

33 
381 
2,620 
1,872 
n/r 
4,431 
238 
3,638 
5,865 


645 
8,900 
1,532 
7,942 
9,608 
4,829 
4,779 

33 

381 
2,620 
1,872 

n/r 
4,431 

238 
3,638 
5,865 


645 
8,900 
1,532 
7,942 
9,608 


4,829 
4,779 


Mean 
736.54 
744.21 
728.79 
730.03 
760.69 
712.07 
716.00 

n/r 
757.25 
748.90 
716.38 
749.01 


684.49 
740.20 
709.15 
741.51 
44.39 
46.17 
42.59 
39.91 
52.87 
35.68 
36.58 
n/r 
51.99 
48.74 
37.06 
48.94 


25.40 
45.73 
34.78 
46.12 
30.54 
33.54 
27.50 


SD 
48.40 
48.34 
47.21 
42.61 
48.96 
40.95 
42.04 

n/r 
44.50 
43.33 
41.02 
48.45 


24.19 
47.51 
42.41 
47.74 
18.36 
18.11 
18.43 
15.64 
19.26 
15.36 
15.71 

n/r 
17.30 
17.00 
15.46 
18.54 


9.21 
18.08 
16.46 
18.13 
14.17 
13.79 
13.90 


Min 
650 
650 
650 
653 
650 
650 
650 

n/r 
650 
650 
650 
650 


650 
650 
650 
650 
10 
10 
10 
14 
10 
10 
10 
n/r 
10 
10 
10 
10 


10 
10 
10 
10 
10 
10 
10 


Max 
850 
850 
850 
811 
850 
850 
850 

n/r 
850 
850 
850 
850 


770 
850 
850 
850 
90 
90 
90 
82 
90 
90 
90 
n/r 
90 
90 
90 
90 


58 
90 
90 
90 
60 
60 
60 
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Group Type 
Ethnicity 


Economic Status" 


English Learner 
Status 


Disabilities 


Group 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


645 
8,900 
1,532 
7,942 


Mean 
30.64 


37.39 
23.74 
25.38 

n/r 
36.04 
34.06 
25.18 
33.83 


16.44 
31.52 
22.83 
31.93 


2017 Technical Report 


SD 
13.53 


13.24 
12.86 
13.06 

n/r 
12.86 
12.57 
12.82 
13.98 


8.89 
13.94 
12.93 
13.95 


10 


60 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP): receipt of 
free or reduced-price lunch (FRL). n/r = not reported due to n<25. 
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Table ADD.12.3 Fall 2016 Subgroup Performance for ELA/L Scale Scores: Grade 11 


Group Type Group 
Full Summative Score 
Gender Female 
Male 
Ethnicity American Indian/Alaska Native 


Economic Status™ 


English Learner 
Status 


Disabilities 


Reading Score 
Gender 


Ethnicity 


Economic Status" 


English Learner 
Status 


Disabilities 


Writing Score 
Gender 
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Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 

Male 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


Female 
Male 


N 
15,342 
6,817 
8,525 
896 
357 
3,132 
5,581 
n/r 
4,781 
285 
7,308 
7,667 


1,122 
13,892 
3,095 
11,903 
15,342 
6,817 
8,525 
896 
357 
3,132 
5,581 
n/r 
4,781 
285 
7,308 
7,667 


1,122 
13,892 

3,095 
11,903 
15,342 


6,817 
8,525 


Mean 
719.57 
726.83 
713.76 
714.67 
739.67 
712.03 
713.45 

n/r 
730.97 
722.29 
712.09 
726.50 


697.53 
721.28 
701.08 
724.20 
38.49 
40.40 
36.97 
35.46 
45.88 
35.54 
36.01 
n/r 
43.30 
39.78 
35.49 
41.33 


29.49 
39.21 
31.54 
40.27 
24.07 
27.35 
21.44 


SD 
35.17 
36.24 
33.16 
27.70 
41.11 
32.64 
31.71 

n/r 
38.08 
34.25 
31.29 
37.25 


24.03 
35.36 
28.80 
35.12 
13.84 
14.15 
13.40 
10.75 
16.33 
12.55 
12.40 

n/r 
15.10 
13.69 
12.21 
14.70 


9.19 
13.91 
11.39 
13.87 
12.39 
12.25 
11.87 


Min 
650 
650 
650 
650 
650 
650 
650 

n/r 
650 
650 
650 
650 


650 
650 
650 
650 


Max 
850 
850 
850 
824 
850 
831 
849 

n/r 
850 
810 
849 
850 


781 
850 
822 
850 
90 
90 
90 
90 
90 
80 
89 
n/r 
90 
79 
90 
90 


61 
90 
82 
90 
60 
60 
60 
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Group Type 
Ethnicity 


Economic Status" 


English Learner 
Status 


Disabilities 


Group 

American Indian/Alaska Native 
Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


357 
3,132 
5,581 

n/r 
4,781 

285 
7,308 
7,667 


1,122 
13,892 
3,095 
11,903 


Mean 
24.14 


30.43 
21.85 
22.53 

n/r 
26.80 
24.64 
22.02 
25.91 


18.39 
24.49 
18.33 
25.47 


2017 Technical Report 


SD 
11.24 


12.68 
11.96 
11.73 

n/r 
12.99 
12.20 
11.72 
12.74 


10.14 
12.46 
10.71 
12.38 


10 


60 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP): receipt of 
free or reduced-price lunch (FRL). n/r = not reported due to n<25. 
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Table ADD.12.4 Fall 2016 Subgroup Performance for Mathematics Scale Scores: Algebra | 


Group Type Group 
Full Summative Score 
Gender Female 
Male 
Ethnicity American Indian/Alaska Native 


Economic Status" 


English Learner 
Status 


Disabilities 


Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


N 

7,159 
3,524 
3,635 
32 
275 
1,952 
1,477 
n/r 
3,126 
172 
2,660 
4,348 


380 
6,657 
1,064 
5,937 


Mean 
736.05 
738.85 
733.33 
724.84 
756.39 
715.57 
724.09 

n/r 
752.80 
747.98 
722.49 
744.92 


702.63 
738.11 
712.48 
740.50 


2017 Technical Report 


SD 
36.66 

35.91 
37.17 
38.66 
32.19 
31.45 
31.91 

n/r 
33.10 
38.03 
32.32 
36.63 


22.41 
36.45 
31.42 
36.06 


Min 

650 
650 
650 
653 
657 
650 
650 
n/r 
650 
650 
650 
650 


650 
650 
650 
650 


Max 

850 
848 
850 
817 
848 
821 
837 
n/r 
850 
819 
831 
850 


800 
850 
827 
850 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP): receipt of 
free or reduced-price lunch (FRL). n/r = not reported due to n<25. 
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2017 Technical Report 


Table ADD.12.5 Fall 2016 Subgroup Performance for Mathematics Scale Scores: Geometry 


Group Type Group 
Full Summative Score 
Gender Female 
Male 
Ethnicity American Indian/Alaska Native 


Economic Status" 


English Learner 
Status 


Disabilities 


Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


N 
6,827 
3,416 
3,411 

195 
258 
1,121 
2,260 
n/r 
2,745 
130 
2,916 
3,690 


363 
6,275 
AAG 
5,392 


Mean 
724.81 
725.95 
723.68 
710.51 
749.70 
717.60 
714.52 

n/r 
735.49 
725.55 
716.68 
731.53 


703.89 
726.06 
710.17 
727.92 


SD 
25.85 
25.65 
26.01 
19.10 
27.08 
21.57 
21.42 

n/r 
25.36 
25.43 
22.16 
26.67 


15.37 
25.80 
20.80 
25.84 


Min 
650 
650 
650 
650 
650 
650 
650 

n/r 
650 
662 
650 
650 


665 
650 
650 
650 


Max 
811 
800 
811 
769 
799 
787 
800 

n/r 
811 
791 
796 
811 


752 
811 
811 
807 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP): receipt of 
free or reduced-price lunch (FRL). n/r = not reported due to n<25. 
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2017 Technical Report 


Table ADD.12.6 Fall 2016 Subgroup Performance for Mathematics Scale Scores: Algebra II 


Group Type Group 
Full Summative Score 
Gender Female 
Male 
Ethnicity American Indian/Alaska Native 


Economic Status™ 


English Learner 
Status 


Disabilities 


Asian 

Black or African American 
Hispanic/Latino 

Native Hawaiian or Pacific Islander 
Multiple Race Selected 

White 

Economically Disadvantaged 


Not Economically Disadvantaged 
English Learner (EL) 


Non English Learner 
Students with Disabilities (SWD) 
Students without Disabilities 


N 
15,647 
7,953 
7,694 
1,034 
509 
2,724 
5,596 
n/r 
5,271 
321 
7,055 
8,336 


390 
14,542 
1,909 
13,436 


Mean 
706.60 
707.61 
705.55 
693.19 
743.16 
694.47 
696.20 

n/r 
722.86 
716.53 
695.77 
715.97 


684.13 
708.09 
689.01 
709.14 


SD 
35.49 
34.04 
36.90 
22.07 
41.58 
28.17 
26.24 

n/r 
39.86 
39.16 
27.24 
39.06 


21.44 
35.79 
28.76 
35.72 


Min 
650 
650 
650 
650 
650 
650 
650 

n/r 
650 
650 
650 
650 


650 
650 
650 
650 


Max 
850 
850 
850 
825 
850 
831 
834 

n/r 
850 
831 
831 
850 


778 
850 
847 
850 


Note: ‘Economic status was based on participation in National School Lunch Program (NSLP): receipt of 
free or reduced-price lunch (FRL). n/r = not reported due to n<25. 
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